# Aveek Sarkar

> Cybersecurity & ML student. Hi, I'm Aveek Sarkar, a second-year B.Tech student at GITAM University, Visakhapatnam, working across cybersecurity and machine learning.

- Interactive portfolio: https://aveeksarkar.pages.dev/
- Plain profile page: https://aveeksarkar.pages.dev/about/
- GitHub: https://github.com/Sarkar-Aveek
- Email: aveeksarkar.official@gmail.com

## About

Hi, I'm Aveek Sarkar, a second-year B.Tech student at GITAM University, Visakhapatnam, working across cybersecurity and machine learning.

I build : GANs that sharpen Sentinel-2 imagery from 10 m to a 2.39 m grid, an optimiser that decides which train waits in which loop, and local LLMs steered by editing their activations. Five flags are hidden on this machine. Type ctf.

Open to internships and research roles.

- **github**: https://github.com/Sarkar-Aveek
- **email**: aveeksarkar.official@gmail.com

## Skills

### Languages

- **Python**: ML pipelines, FastAPI services, data collection, tooling
- **C**: low-latency video streaming between a PC and a Raspberry Pi
- **C++**: Arduino, ESP32 and ESP8266 firmware
- **JavaScript**: web apps, three.js, WebGL2 (this site)
- **GLSL**: CRT phosphor, glow and barrel-distortion shaders
- **SQL**: SQLite, PostgreSQL
- **Bash**: Raspberry Pi installers, systemd services

### Machine learning

Deep learning PyTorch, PyTorch Lightning, torchvision, timm, Hugging Face (transformers, Hub, safetensors)
- **Generative**: ESRGAN / BasicSR fine-tuning, multi-date super-resolution, LPIPS, OpenCLIP
- **Classical ML**: scikit-learn (random forests), joblib
Preprocessing signal smoothing (One Euro filter), normalisation, feature engineering
- **LLMs**: llama.cpp, Ollama, GGUF quantisation, control vectors (activation steering), tool calling
- **Speech**: Silero VAD, Parakeet STT (sherpa-onnx), Kokoro TTS, pyttsx3
- **Runtime**: ONNX Runtime, CUDA
- **Evaluation**: edge-F1, LPIPS, opensr-test, GSM8K, BFCL, XSTest
- **Tracking**: Weights & Biases, TensorBoard

### Data and vision

- **Data**: NumPy, pandas, SciPy, Matplotlib
- **Vision**: OpenCV, scikit-image, Pillow, Kornia, MediaPipe, Ultralytics YOLO
- **Geospatial**: rasterio, pyproj, Shapely, GeoTIFF, Sentinel-2 L2A, Copernicus Data Space

### Systems

- **Optimisation**: Google OR-Tools CP-SAT
- **Backend**: FastAPI, Pydantic, Uvicorn, WebSocket, SQLite, PostgreSQL, Redis, Docker
- **Testing**: pytest, Playwright
- **Maps**: OpenStreetMap / Overpass, Leaflet, Chart.js
- **Deploy**: Cloudflare Pages and Tunnel, Git, Git LFS
- **Hardware**: Raspberry Pi, ESP32-CAM, ESP8266, Arduino, TETRIX PRIZM, GPS modules

### Security

- **Web**: OWASP Top 10, SQLi, XSS, SSRF, IDOR
- **Network**: Nmap, Wireshark, tcpdump
- **Linux**: permissions, hardening, privilege escalation
- **Crypto**: hashing, salting, JWTs
- **CTF**: this site has five flags: type ctf

## Projects

### VIYGAN-2

Multi-temporal generative super-resolution mapping (SRM) for Sentinel-2 L2A over the Indian subcontinent. Smart India Hackathon problem SIH26142, NTRO.

Architecture
- Multi-date ESRGAN (RRDBNet generator, 16.7 M params) ingesting an 8-epoch stack of co-registered L2A acquisitions as a 24-channel tensor; 4x upscale from a 10 m GSD to a 2.39 m output grid
- Transfer learning from Allen AI's Satlas checkpoint, adversarially fine-tuned on 103,936 paired patches from 55 AOIs against zoom-17 ArcGIS World Imagery; EMA generator weights
- Two production heads: arcgis_B (basemap radiometry) and s2colour, which enforces spectral fidelity through an in-loop low-pass colour lock per 10 m cell instead of post-hoc histogram matching
- Per-pixel confidence / uncertainty map flagging hallucinated high-frequency detail

Evaluation (7 held-out AOIs never seen in training, 2,800 tiles)
- Edge-F1 0.642 and LPIPS 0.182 vs 0.524 / 0.382 for the Satlas baseline; beats ESA SEN2SR, LDSR-S2 (latent diffusion) and Real-ESRGAN on structure and perceptual metrics in every biome
- Geometric consistency: 0.010 px sub-pixel drift against the multi-date consensus; inputs co-registered to 0.027 px
- PSNR deliberately not the headline: pixel-wise metrics reward blur, which is documented with literature citations

Serving
- FastAPI inference backend, on-demand AOI fetch from Copernicus Data Space, GPU SR into a WebP XYZ tile pyramid, georeferenced GeoTIFF export (EPSG:3857)
- Web GIS front end: spectral indices, zonal statistics, edge detection, provenance; static edge on Cloudflare Pages, origin via Cloudflare Tunnel, public read-only and private compute instances

Model card: ~/models/viygan-2.md
- **Stack**: Python, PyTorch, ESRGAN, FastAPI, SQLite, Leaflet, Cloudflare
- **App**: https://viygan-2.pages.dev
- **Code**: https://github.com/Sarkar-Aveek/VIYGAN-2

### rail-controller

Real-time decision-support system (DSS) for Indian Railways section control: constraint-programming optimisation of train precedence and crossings on any corridor, built on demand.

Data platform
- Ingests the nationwide DataMeet timetable (8,966 stations, 5,208 trains, 417,080 schedule rows, 12,019 track links) into an immutable in-memory catalog
- Data-driven corridor construction: dominant-route extraction across trains calling at both ends, falling back to Dijkstra over a national track graph weighted by traffic density; plausibility guards refuse fabricated sections
- National OSM rail graph built offline from Overpass, with station snapping and per-block polyline routing precomputed into a geometry store; runtime attach is O(1) per block
- Kinematic run-time model: per-block technical minimum from sectional and class speed ceilings, timetable recovery margin redistributed proportionally, eliminating quantisation artefacts of up to 474 km/h

Optimiser (Google OR-Tools CP-SAT)
- Integer entry/exit decision variables per train per block; AddNoOverlap for line occupancy per running line and direction with headway; AddCumulative for platform and loop capacity
- Priority-weighted tardiness objective, warm-started from a greedy FCFS baseline, with staged constraint relaxation when infeasible
- Plan diffed against the baseline into actionable PRECEDENCE and HOLD recommendations with net train-minute gain; requests are time-shifted so advice lands still actionable
- Measured with a 10 s budget on a 240 min horizon: average delay 47.90 -> 35.14 min on an 80-station corridor (2,337 intervals), optimal in 3.5 s on short sections

Simulation
- Discrete-time simulation driven by measured wall-clock (under 1 % drift at 60x), admission windowing that cuts the pushed state from 336 KB to about 100 KB on 2,000 km corridors
- Disruption injection (TSR, block closure, late entry, breakdown) and what-if scenario forking with side-by-side KPIs; live state over WebSocket

- **Stack**: Python, FastAPI, OR-Tools CP-SAT, OpenStreetMap / Overpass, WebSocket

### KalJahaz

Predictive chartering and freight-cost forecasting for dry-bulk tonnage into India's East Coast ports. Smart India Hackathon.

Market model
- Online adaptive recursive least squares (RLS) with a forgetting factor, one per vessel class, regressed on the class's own Baltic sub-index (BCI, BPI, BSI, BHSI; BDI as fallback) plus China PMI, India IIP, iron-ore input values and world trade volume
- Continuously benchmarked against a naive no-change forecaster via EWMA squared error
- Multi-horizon forecasts of total voyage cost at T+0, +7, +30 and +60 days with the optimal fixing window

Voyage economics
- Routes decomposed into great-circle legs between lane waypoints (chokepoints, capes, straits) approximating AIS traffic lanes
- Involuntary speed loss from wind and sea state via Kwon's method, ocean currents, tidal windows and port congestion, feeding days-at-sea and total cost
- Port physical constraints (LOA, beam, draft, handling rate) gate permissible vessel and DWT combinations

Data pipeline
- Live ingestion every 6 h from public Baltic index scrapes, IMF via FRED and NBS via DBnomics; Open-Meteo weather with monsoon and cyclone climatology fallback; per-series provenance and freshness surfaced in the UI

- **Stack**: Python (zero-dependency stdlib server), Leaflet, Chart.js, Cloudflare Pages + Tunnel
- **Site**: https://kaljahaz.pages.dev

### ml-preprocessing

A data cleaning and normalisation pipeline that turns noisy, inconsistent raw records into clean, invariant features any ML model can learn from. The same code prepares the training dataset and every live sample, so training and inference never drift apart. Sign language to speech is the demo it was built and measured on.

Data cleaning
- Denoising: an adaptive One Euro low-pass filter on every coordinate of every record before it is stored, removing jitter without smearing fast, real changes
- Numerical safety: degenerate records (all points collapsed) and zero-length vectors are guarded, so no NaN or infinity ever reaches the model
- Duplicate suppression: a held sample is not re-emitted on every frame, only after a real change

Normalisation and feature engineering
- Translation normalisation: every record re-centred on a reference point
- Scale normalisation: divided by a reference length, so size and distance from the sensor drop out
- Rotation invariance: in-plane tilt does not change any feature
- Raw 63-value records reduced to 36 engineered features (lengths, joint angles, pairwise distances, orientation): compact, model-agnostic and far easier to learn from
- Every invariance is unit-tested: shifting, resizing or tilting the input must leave the features unchanged
- Stratified train/test split so every class is represented in evaluation

Input
- Raw 21-point 3D landmarks from MediaPipe over an MJPG camera stream (921,600 pixel values per frame reduced to 63 numbers, then 36)

Demo: sign language to speech
- Random-forest ensemble (100 trees) over 44 sign classes, trained on 2,526 self-collected samples
- 2.15 ms per prediction single-threaded; parallel inference measured slower (20.6 ms) and rejected
- Stability gate: a prediction is only accepted once it holds steady, so output does not flicker
- Predictions assembled into sentences and spoken with text-to-speech without freezing the video
- The same code ships to a Raspberry Pi via a setup script

Model card: ~/models/sign-language-demo.md
- **Stack**: Python, MediaPipe, OpenCV, NumPy, pandas, scikit-learn, joblib, pyttsx3, Raspberry Pi

### alfred

Fully offline, agentic voice assistant on a local 9B LLM, running on a single 8 GB laptop GPU.

Inference
- Quantised Qwen-family 9B (Q4_K_M GGUF) served via llama.cpp and Ollama with flash attention and an 8-bit KV cache for a 32k context; vision through a multimodal projector
- Persona via representation engineering: a learned control vector injected into the residual stream (activation steering), no system prompt and no fine-tuning
- Optional cloud-model routing with circuit-breaker fallback to the local model on timeout

Speech stack
- Streaming audio in 160 ms frames: Silero VAD, name-based wake detection with fuzzy matching, loudness gating to the headset wearer
- Parakeet-TDT 0.6B (int8, sherpa-onnx) ASR at about 80 ms, chosen over Whisper distil (0.9 s) and streaming Zipformer (6-9 % WER)
- Kokoro ONNX TTS with sentence-level streaming off the SSE token stream, and barge-in interruption
- Tool calling: app and window control, web search, screen understanding, file I/O, keyboard and media control, reminders

Research: ~/research/activation-steering.md
- **Stack**: Python, llama.cpp, Ollama, transformers, sherpa-onnx, ONNX Runtime

### NIP

Low-latency, zero-configuration screen streaming from a Windows PC to a display driven by any Raspberry Pi. Written in C.

Sender (Windows)
- Capability probing with graceful degradation: DXGI Desktop Duplication capture (GDI fallback), hardware encoder auto-selection NVENC > QuickSync > AMF > x264, each validated by a test encode
- Low-latency encode profile: no B-frames, intra-refresh, single-frame VBV buffer; codec negotiated to what the Pi decodes in hardware (H.264 on Zero to Pi 4, HEVC on Pi 5)

Receiver (Raspberry Pi)
- Hardware decode straight to KMS scan-out with no compositor, no jitter buffer and sync disabled
- Performance CPU governor, enlarged socket buffers, raised scheduling priority; systemd service with crash recovery
- Transports: LAN, the Pi as its own Wi-Fi access point, or USB gadget Ethernet; peer discovery across every adapter with link preference USB > Ethernet > Wi-Fi
- Design target 20-40 ms glass-to-glass at 1080p60; designed next step is a raw Layer-2 Ethernet transport (custom EtherType) with FEC instead of retransmission

- **Stack**: C, ffmpeg, GStreamer, DXGI, V4L2 / KMS, systemd

### hardware

Embedded and edge-vision builds.

- ESP32-CAM firmware streaming MJPEG over Wi-Fi in station and soft-AP modes
- Off-board YOLO object detection (Ultralytics YOLOv8 / YOLO11) inferring on the live ESP32-CAM stream
- GNSS tracking with TinyGPS++, a Bluetooth-controlled rover, ESP8266 servo actuation and an SPI TFT display
- Robotics work on the TETRIX PRIZM controller

- **Stack**: C++ (Arduino), ESP32, ESP8266, Python, Ultralytics YOLO

### crt-terminal

This site: a functional POSIX-style shell rendered through a physically inspired CRT pipeline.

- three.js / WebGL2 scene with the static room baked once into half-float render targets; per frame only the tube, button and LED are redrawn, so it holds frame rate on a Radeon 610M iGPU
- Custom GLSL: phosphor persistence, bloom from a mip chain, scanlines, barrel distortion, in-shader tonemapping
- Shell over an in-memory VFS with pipes, &&, regex grep, find, tree, hidden dotfiles and per-session chmod
- Real client-side crypto tooling (SHA-256, MD5, base64, xxd, JWT decode, hash identification), a five-flag CTF verified by hash, and a from-scratch MLP trained live
- Build-time SEO/AEO: JSON-LD, llms.txt, crawlable profile, strict CSP on Cloudflare Pages

- **Stack**: JavaScript, three.js, WebGL2, GLSL, Vite

## Machine learning models

### viygan-2 (arcgis_B and s2colour)

- **Task**: multi-temporal single-image-out super-resolution, 4x, 10 m GSD to a 2.39 m grid
- **Architecture**: ESRGAN, RRDBNet generator (24 in-channels = 8 epochs x RGB), 16.7 M params, EMA weights
- **Training**: adversarial + perceptual fine-tuning from Satlas; s2colour trained with an in-loop low-pass colour lock
- **Data**: 103,936 paired patches, 55 Indian AOIs, reference ArcGIS World Imagery at 2.39 m/px (zoom 17)
- **Test**: 7 held-out AOIs across urban, paddy, desert, forest, hills and snow; 2,800 tiles
- **Metrics**: arcgis_B edge-F1 0.642, LPIPS 0.182; s2colour edge-F1 0.638, LPIPS 0.193
- **Baselines**: Satlas 0.524 / 0.382, ESA LDSR-S2 0.492 / 0.349, Real-ESRGAN 0.458 / 0.440, SEN2SR 0.457 / 0.484
- **Geometry**: 0.010 px drift vs multi-date consensus (s2colour); inputs co-registered to 0.027 px
- **Limits**: output is a 2.39 m grid, not observed resolution; sub-metre detail is inferred, so new tiles are validated against their Sentinel-2 input and the confidence map only

### sign-language-demo (built on ml-preprocessing)

- **Task**: 44-class static hand-sign classification from 3D landmarks
- **Data**: 2,526 self-collected samples, 50-100 per class
- **Features**: 36 normalised geometric features over 21 MediaPipe landmarks, One Euro filtered
- **Architecture**: random forest, 100 trees, 40,938 nodes, depth 13-19 (mean 15.8)
- **Inference**: 2.15 ms single-threaded (n_jobs=1); about 870 split decisions per prediction

## Research

### Radiometrically Constrained Multi-Temporal GAN Super-Resolution of Sentinel-2 Imagery over India

- **Author**: Aveek Sarkar
- **Problem**: 10 m Sentinel-2 bands blur roads, small buildings and smallholder field boundaries, and existing GAN super-resolution models are trained on US imagery and copy the reference's colours instead of Sentinel-2's measured reflectance
- **Method**: Satlas eight-date ESRGAN fine-tuned on 103,936 L2A pairs from 55 Indian sites (arcgis_B); s2colour adds a parameter-free band-value constraint inside the generator, trained with colour-invariant losses and a high-pass discriminator
- **Results**: 7 held-out sites, 2,800 tiles: edge-F1 0.524 -> 0.642 and LPIPS 0.382 -> 0.182 vs Satlas; both models beat SEN2SR, LDSR-S2 and Real-ESRGAN at every site; s2colour cuts the reflectance error vs its input from 0.095 to 0.022 at edge-F1 0.638
- **Also**: 3.01 dB upper bound on the PSNR penalty of a posterior-sampling estimator; validation against same-day VENuS reflectance; per-pixel confidence layer with AUROC 0.743

- **Paper**: multitemporal-gan-super-resolution.pdf

### Persona control via activation steering

- **Question**: can an LLM adopt a stylistic persona with no system prompt and no weight updates?
- **Method**: contrastive activation addition: 300 Jeeves lines (P. G. Wodehouse) paired with semantically matched neutral replies; per-layer steering direction = difference of mean residual-stream activations, exported as a llama.cpp control vector and injected into layers 8-24 of 32
- **Eval**: public benchmarks only: XSTest (compliance), GSM8K (reasoning), BFCL (function calling)
- **Result**: at coefficient 0.1 the persona holds with no capability loss: GSM8K 90 -> 93, BFCL 87 -> 90; 0.2 drifts into archaic register, 0.3 degenerates; the dose-response curve is the finding

### Autonomous continuous cognition (ongoing)

- **Question**: what does a local LLM do when it is never prompted and never stops generating?
- **Setup**: pretrained 9B hybrid Gated-DeltaNet / attention model, no chat template, assistant direction ablated; episodic memory in a graph instead of a context window; a background subconscious process; a sandboxed world of offline Wikipedia with mic input and self-initiated TTS
- **Status**: in progress; earlier ARC-AGI architecture work paused pending HPC access

## Resume

- **Cybersecurity & ML student**: |  aveeksarkar.official@gmail.com

#### Education

- **B.Tech, GITAM University, Visakhapatnam**: 2025 - 2029 (2nd year)

#### Projects

- VIYGAN-2: multi-date GAN super-resolution of Sentinel-2 for India, 10 m to 2.39 m (SIH26142, NTRO)
- rail-controller: CP-SAT train precedence and crossing decisions on any Indian Railways corridor
- KalJahaz: dry-bulk freight cost forecasting for India's East Coast ports (Smart India Hackathon)
- ml-preprocessing: data cleaning and normalisation pipeline for any ML model (denoising, NaN-safe handling, translation / scale / rotation normalisation, feature engineering, unit-tested invariances); demo: 44 hand signs to speech at 2.15 ms per prediction on a Raspberry Pi
- alfred: offline voice assistant on a local 9B model, personality by activation steering
- NIP: low-latency PC to Raspberry Pi screen streaming in C
- always-thinking mind: a never-prompted local LLM with a memory graph (ongoing research)
- hardware: ESP32-CAM streaming with YOLO detection, GPS, a Bluetooth car

#### Full version

- **resume.pdf**: one page, ready to download

## Contact

The fastest way to reach me is email.

- **github**: https://github.com/Sarkar-Aveek
- **email**: aveeksarkar.official@gmail.com

Or type: open email

## Questions and answers

### Who is Aveek Sarkar?

Aveek Sarkar is a Cybersecurity & ML student. Hi, I'm Aveek Sarkar, a second-year B.Tech student at GITAM University, Visakhapatnam, working across cybersecurity and machine learning.

### What are Aveek Sarkar's skills?

Aveek Sarkar works with Cybersecurity, Machine learning, Deep learning, Generative adversarial networks, Satellite image super-resolution, Remote sensing, Large language models, Activation steering, Constraint programming, Computer vision, Python, PyTorch, FastAPI, C, Embedded systems, Linux.

### What projects has Aveek Sarkar built?

VIYGAN-2: Multi-temporal generative super-resolution mapping (SRM) for Sentinel-2 L2A over the Indian subcontinent. Smart India Hackathon problem SIH26142, NTRO. rail-controller: Real-time decision-support system (DSS) for Indian Railways section control: constraint-programming optimisation of train precedence and crossings on any corridor, built on demand. KalJahaz: Predictive chartering and freight-cost forecasting for dry-bulk tonnage into India's East Coast ports. Smart India Hackathon. ml-preprocessing: A data cleaning and normalisation pipeline that turns noisy, inconsistent raw records into clean, invariant features any ML model can learn from. The same code prepares the training dataset and every live sample, so training and inference never drift apart. Sign language to speech is the demo it was built and measured on. alfred: Fully offline, agentic voice assistant on a local 9B LLM, running on a single 8 GB laptop GPU. NIP: Low-latency, zero-configuration screen streaming from a Windows PC to a display driven by any Raspberry Pi. Written in C. hardware: Embedded and edge-vision builds. crt-terminal: This site: a functional POSIX-style shell rendered through a physically inspired CRT pipeline.

### Is Aveek Sarkar available for work?

Aveek Sarkar is open to internships and research roles.

### How can I contact Aveek Sarkar?

Reach Aveek Sarkar by email aveeksarkar.official@gmail.com, GitHub https://github.com/Sarkar-Aveek.