Aveek Sarkar

Hi, I'm Aveek Sarkar, a second-year B.Tech student at GITAM University, Visakhapatnam, working across cybersecurity and machine learning.

I build : GANs that sharpen Sentinel-2 imagery from 10 m to a 2.39 m grid, an optimiser that decides which train waits in which loop, and local LLMs steered by editing their activations. Five flags are hidden on this machine. Type ctf.

Open to internships and research roles.

github

https://github.com/Sarkar-Aveek

email

aveeksarkar.official@gmail.com

Languages

Python

ML pipelines, FastAPI services, data collection, tooling

C

low-latency video streaming between a PC and a Raspberry Pi

C++

Arduino, ESP32 and ESP8266 firmware

JavaScript

web apps, three.js, WebGL2 (this site)

GLSL

CRT phosphor, glow and barrel-distortion shaders

SQL

SQLite, PostgreSQL

Bash

Raspberry Pi installers, systemd services

Machine learning

Deep learning PyTorch, PyTorch Lightning, torchvision, timm, Hugging Face (transformers, Hub, safetensors)

Generative ESRGAN / BasicSR fine-tuning, multi-date super-resolution, LPIPS, OpenCLIP

Classical ML scikit-learn (random forests), joblib

Preprocessing signal smoothing (One Euro filter), normalisation, feature engineering

LLMs llama.cpp, Ollama, GGUF quantisation, control vectors (activation steering), tool calling

Speech Silero VAD, Parakeet STT (sherpa-onnx), Kokoro TTS, pyttsx3

Runtime ONNX Runtime, CUDA

Evaluation edge-F1, LPIPS, opensr-test, GSM8K, BFCL, XSTest

Tracking Weights & Biases, TensorBoard

Data and vision

Data

NumPy, pandas, SciPy, Matplotlib

Vision

OpenCV, scikit-image, Pillow, Kornia, MediaPipe, Ultralytics YOLO

Geospatial

rasterio, pyproj, Shapely, GeoTIFF, Sentinel-2 L2A, Copernicus Data Space

Systems

Optimisation

Google OR-Tools CP-SAT

Backend

FastAPI, Pydantic, Uvicorn, WebSocket, SQLite, PostgreSQL, Redis, Docker

Testing

pytest, Playwright

Maps

OpenStreetMap / Overpass, Leaflet, Chart.js

Deploy

Cloudflare Pages and Tunnel, Git, Git LFS

Hardware

Raspberry Pi, ESP32-CAM, ESP8266, Arduino, TETRIX PRIZM, GPS modules

Security

Web

OWASP Top 10, SQLi, XSS, SSRF, IDOR

Network

Nmap, Wireshark, tcpdump

Linux

permissions, hardening, privilege escalation

Crypto

hashing, salting, JWTs

CTF

this site has five flags: type ctf

VIYGAN-2

Multi-temporal generative super-resolution mapping (SRM) for Sentinel-2 L2A over the Indian subcontinent. Smart India Hackathon problem SIH26142, NTRO.

Architecture

• Multi-date ESRGAN (RRDBNet generator, 16.7 M params) ingesting an 8-epoch stack of co-registered L2A acquisitions as a 24-channel tensor; 4x upscale from a 10 m GSD to a 2.39 m output grid

• Transfer learning from Allen AI's Satlas checkpoint, adversarially fine-tuned on 103,936 paired patches from 55 AOIs against zoom-17 ArcGIS World Imagery; EMA generator weights

• Two production heads: arcgis_B (basemap radiometry) and s2colour, which enforces spectral fidelity through an in-loop low-pass colour lock per 10 m cell instead of post-hoc histogram matching

• Per-pixel confidence / uncertainty map flagging hallucinated high-frequency detail

Evaluation (7 held-out AOIs never seen in training, 2,800 tiles)

• Edge-F1 0.642 and LPIPS 0.182 vs 0.524 / 0.382 for the Satlas baseline; beats ESA SEN2SR, LDSR-S2 (latent diffusion) and Real-ESRGAN on structure and perceptual metrics in every biome

• Geometric consistency: 0.010 px sub-pixel drift against the multi-date consensus; inputs co-registered to 0.027 px

• PSNR deliberately not the headline: pixel-wise metrics reward blur, which is documented with literature citations

Serving

• FastAPI inference backend, on-demand AOI fetch from Copernicus Data Space, GPU SR into a WebP XYZ tile pyramid, georeferenced GeoTIFF export (EPSG:3857)

• Web GIS front end: spectral indices, zonal statistics, edge detection, provenance; static edge on Cloudflare Pages, origin via Cloudflare Tunnel, public read-only and private compute instances

Model card: ~/models/viygan-2.md

Stack Python, PyTorch, ESRGAN, FastAPI, SQLite, Leaflet, Cloudflare

App https://viygan-2.pages.dev

Code https://github.com/Sarkar-Aveek/VIYGAN-2

rail-controller

Real-time decision-support system (DSS) for Indian Railways section control: constraint-programming optimisation of train precedence and crossings on any corridor, built on demand.

Data platform

• Ingests the nationwide DataMeet timetable (8,966 stations, 5,208 trains, 417,080 schedule rows, 12,019 track links) into an immutable in-memory catalog

• Data-driven corridor construction: dominant-route extraction across trains calling at both ends, falling back to Dijkstra over a national track graph weighted by traffic density; plausibility guards refuse fabricated sections

• National OSM rail graph built offline from Overpass, with station snapping and per-block polyline routing precomputed into a geometry store; runtime attach is O(1) per block

• Kinematic run-time model: per-block technical minimum from sectional and class speed ceilings, timetable recovery margin redistributed proportionally, eliminating quantisation artefacts of up to 474 km/h

Optimiser (Google OR-Tools CP-SAT)

• Integer entry/exit decision variables per train per block; AddNoOverlap for line occupancy per running line and direction with headway; AddCumulative for platform and loop capacity

• Priority-weighted tardiness objective, warm-started from a greedy FCFS baseline, with staged constraint relaxation when infeasible

• Plan diffed against the baseline into actionable PRECEDENCE and HOLD recommendations with net train-minute gain; requests are time-shifted so advice lands still actionable

• Measured with a 10 s budget on a 240 min horizon: average delay 47.90 -> 35.14 min on an 80-station corridor (2,337 intervals), optimal in 3.5 s on short sections

Simulation

• Discrete-time simulation driven by measured wall-clock (under 1 % drift at 60x), admission windowing that cuts the pushed state from 336 KB to about 100 KB on 2,000 km corridors

• Disruption injection (TSR, block closure, late entry, breakdown) and what-if scenario forking with side-by-side KPIs; live state over WebSocket

Stack

Python, FastAPI, OR-Tools CP-SAT, OpenStreetMap / Overpass, WebSocket

KalJahaz

Predictive chartering and freight-cost forecasting for dry-bulk tonnage into India's East Coast ports. Smart India Hackathon.

Market model

• Online adaptive recursive least squares (RLS) with a forgetting factor, one per vessel class, regressed on the class's own Baltic sub-index (BCI, BPI, BSI, BHSI; BDI as fallback) plus China PMI, India IIP, iron-ore input values and world trade volume

• Continuously benchmarked against a naive no-change forecaster via EWMA squared error

• Multi-horizon forecasts of total voyage cost at T+0, +7, +30 and +60 days with the optimal fixing window

Voyage economics

• Routes decomposed into great-circle legs between lane waypoints (chokepoints, capes, straits) approximating AIS traffic lanes

• Involuntary speed loss from wind and sea state via Kwon's method, ocean currents, tidal windows and port congestion, feeding days-at-sea and total cost

• Port physical constraints (LOA, beam, draft, handling rate) gate permissible vessel and DWT combinations

Data pipeline

• Live ingestion every 6 h from public Baltic index scrapes, IMF via FRED and NBS via DBnomics; Open-Meteo weather with monsoon and cyclone climatology fallback; per-series provenance and freshness surfaced in the UI

Stack

Python (zero-dependency stdlib server), Leaflet, Chart.js, Cloudflare Pages + Tunnel

Site

https://kaljahaz.pages.dev

ml-preprocessing

A data cleaning and normalisation pipeline that turns noisy, inconsistent raw records into clean, invariant features any ML model can learn from. The same code prepares the training dataset and every live sample, so training and inference never drift apart. Sign language to speech is the demo it was built and measured on.

Data cleaning

• Denoising: an adaptive One Euro low-pass filter on every coordinate of every record before it is stored, removing jitter without smearing fast, real changes

• Numerical safety: degenerate records (all points collapsed) and zero-length vectors are guarded, so no NaN or infinity ever reaches the model

• Duplicate suppression: a held sample is not re-emitted on every frame, only after a real change

Normalisation and feature engineering

• Translation normalisation: every record re-centred on a reference point

• Scale normalisation: divided by a reference length, so size and distance from the sensor drop out

• Rotation invariance: in-plane tilt does not change any feature

• Raw 63-value records reduced to 36 engineered features (lengths, joint angles, pairwise distances, orientation): compact, model-agnostic and far easier to learn from

• Every invariance is unit-tested: shifting, resizing or tilting the input must leave the features unchanged

• Stratified train/test split so every class is represented in evaluation

Input

• Raw 21-point 3D landmarks from MediaPipe over an MJPG camera stream (921,600 pixel values per frame reduced to 63 numbers, then 36)

Demo: sign language to speech

• Random-forest ensemble (100 trees) over 44 sign classes, trained on 2,526 self-collected samples

• 2.15 ms per prediction single-threaded; parallel inference measured slower (20.6 ms) and rejected

• Stability gate: a prediction is only accepted once it holds steady, so output does not flicker

• Predictions assembled into sentences and spoken with text-to-speech without freezing the video

• The same code ships to a Raspberry Pi via a setup script

Model card: ~/models/sign-language-demo.md

Stack Python, MediaPipe, OpenCV, NumPy, pandas, scikit-learn, joblib, pyttsx3, Raspberry Pi

alfred

Fully offline, agentic voice assistant on a local 9B LLM, running on a single 8 GB laptop GPU.

Inference

• Quantised Qwen-family 9B (Q4_K_M GGUF) served via llama.cpp and Ollama with flash attention and an 8-bit KV cache for a 32k context; vision through a multimodal projector

• Persona via representation engineering: a learned control vector injected into the residual stream (activation steering), no system prompt and no fine-tuning

• Optional cloud-model routing with circuit-breaker fallback to the local model on timeout

Speech stack

• Streaming audio in 160 ms frames: Silero VAD, name-based wake detection with fuzzy matching, loudness gating to the headset wearer

• Parakeet-TDT 0.6B (int8, sherpa-onnx) ASR at about 80 ms, chosen over Whisper distil (0.9 s) and streaming Zipformer (6-9 % WER)

• Kokoro ONNX TTS with sentence-level streaming off the SSE token stream, and barge-in interruption

• Tool calling: app and window control, web search, screen understanding, file I/O, keyboard and media control, reminders

Research: ~/research/activation-steering.md

Stack Python, llama.cpp, Ollama, transformers, sherpa-onnx, ONNX Runtime

NIP

Low-latency, zero-configuration screen streaming from a Windows PC to a display driven by any Raspberry Pi. Written in C.

Sender (Windows)

• Capability probing with graceful degradation: DXGI Desktop Duplication capture (GDI fallback), hardware encoder auto-selection NVENC > QuickSync > AMF > x264, each validated by a test encode

• Low-latency encode profile: no B-frames, intra-refresh, single-frame VBV buffer; codec negotiated to what the Pi decodes in hardware (H.264 on Zero to Pi 4, HEVC on Pi 5)

Receiver (Raspberry Pi)

• Hardware decode straight to KMS scan-out with no compositor, no jitter buffer and sync disabled

• Performance CPU governor, enlarged socket buffers, raised scheduling priority; systemd service with crash recovery

• Transports: LAN, the Pi as its own Wi-Fi access point, or USB gadget Ethernet; peer discovery across every adapter with link preference USB > Ethernet > Wi-Fi

• Design target 20-40 ms glass-to-glass at 1080p60; designed next step is a raw Layer-2 Ethernet transport (custom EtherType) with FEC instead of retransmission

Stack

C, ffmpeg, GStreamer, DXGI, V4L2 / KMS, systemd

hardware

Embedded and edge-vision builds.

• ESP32-CAM firmware streaming MJPEG over Wi-Fi in station and soft-AP modes
• Off-board YOLO object detection (Ultralytics YOLOv8 / YOLO11) inferring on the live ESP32-CAM stream
• GNSS tracking with TinyGPS++, a Bluetooth-controlled rover, ESP8266 servo actuation and an SPI TFT display
• Robotics work on the TETRIX PRIZM controller

Stack

C++ (Arduino), ESP32, ESP8266, Python, Ultralytics YOLO

crt-terminal

This site: a functional POSIX-style shell rendered through a physically inspired CRT pipeline.

• three.js / WebGL2 scene with the static room baked once into half-float render targets; per frame only the tube, button and LED are redrawn, so it holds frame rate on a Radeon 610M iGPU
• Custom GLSL: phosphor persistence, bloom from a mip chain, scanlines, barrel distortion, in-shader tonemapping
• Shell over an in-memory VFS with pipes, &&, regex grep, find, tree, hidden dotfiles and per-session chmod
• Real client-side crypto tooling (SHA-256, MD5, base64, xxd, JWT decode, hash identification), a five-flag CTF verified by hash, and a from-scratch MLP trained live
• Build-time SEO/AEO: JSON-LD, llms.txt, crawlable profile, strict CSP on Cloudflare Pages

Stack

JavaScript, three.js, WebGL2, GLSL, Vite

viygan-2 (arcgis_B and s2colour)

Task

multi-temporal single-image-out super-resolution, 4x, 10 m GSD to a 2.39 m grid

Architecture

ESRGAN, RRDBNet generator (24 in-channels = 8 epochs x RGB), 16.7 M params, EMA weights

Training

adversarial + perceptual fine-tuning from Satlas; s2colour trained with an in-loop low-pass colour lock

Data

103,936 paired patches, 55 Indian AOIs, reference ArcGIS World Imagery at 2.39 m/px (zoom 17)

Test

7 held-out AOIs across urban, paddy, desert, forest, hills and snow; 2,800 tiles

Metrics

arcgis_B edge-F1 0.642, LPIPS 0.182; s2colour edge-F1 0.638, LPIPS 0.193

Baselines

Satlas 0.524 / 0.382, ESA LDSR-S2 0.492 / 0.349, Real-ESRGAN 0.458 / 0.440, SEN2SR 0.457 / 0.484

Geometry

0.010 px drift vs multi-date consensus (s2colour); inputs co-registered to 0.027 px

Limits

output is a 2.39 m grid, not observed resolution; sub-metre detail is inferred, so new tiles are validated against their Sentinel-2 input and the confidence map only

sign-language-demo (built on ml-preprocessing)

Task

44-class static hand-sign classification from 3D landmarks

Data

2,526 self-collected samples, 50-100 per class

Features

36 normalised geometric features over 21 MediaPipe landmarks, One Euro filtered

Architecture

random forest, 100 trees, 40,938 nodes, depth 13-19 (mean 15.8)

Inference

2.15 ms single-threaded (n_jobs=1); about 870 split decisions per prediction

Radiometrically Constrained Multi-Temporal GAN Super-Resolution of Sentinel-2 Imagery over India

Author

Aveek Sarkar

Problem

10 m Sentinel-2 bands blur roads, small buildings and smallholder field boundaries, and existing GAN super-resolution models are trained on US imagery and copy the reference's colours instead of Sentinel-2's measured reflectance

Method

Satlas eight-date ESRGAN fine-tuned on 103,936 L2A pairs from 55 Indian sites (arcgis_B); s2colour adds a parameter-free band-value constraint inside the generator, trained with colour-invariant losses and a high-pass discriminator

Results

7 held-out sites, 2,800 tiles: edge-F1 0.524 -> 0.642 and LPIPS 0.382 -> 0.182 vs Satlas; both models beat SEN2SR, LDSR-S2 and Real-ESRGAN at every site; s2colour cuts the reflectance error vs its input from 0.095 to 0.022 at edge-F1 0.638

Also

3.01 dB upper bound on the PSNR penalty of a posterior-sampling estimator; validation against same-day VENuS reflectance; per-pixel confidence layer with AUROC 0.743

Paper

multitemporal-gan-super-resolution.pdf

Persona control via activation steering

Question

can an LLM adopt a stylistic persona with no system prompt and no weight updates?

Method

contrastive activation addition: 300 Jeeves lines (P. G. Wodehouse) paired with semantically matched neutral replies; per-layer steering direction = difference of mean residual-stream activations, exported as a llama.cpp control vector and injected into layers 8-24 of 32

Eval

public benchmarks only: XSTest (compliance), GSM8K (reasoning), BFCL (function calling)

Result

at coefficient 0.1 the persona holds with no capability loss: GSM8K 90 -> 93, BFCL 87 -> 90; 0.2 drifts into archaic register, 0.3 degenerates; the dose-response curve is the finding

Autonomous continuous cognition (ongoing)

Question

what does a local LLM do when it is never prompted and never stops generating?

Setup

pretrained 9B hybrid Gated-DeltaNet / attention model, no chat template, assistant direction ablated; episodic memory in a graph instead of a context window; a background subconscious process; a sandboxed world of offline Wikipedia with mic input and self-initiated TTS

Status

in progress; earlier ARC-AGI architecture work paused pending HPC access

Cybersecurity & ML student

| aveeksarkar.official@gmail.com

Education

B.Tech, GITAM University, Visakhapatnam

2025 - 2029 (2nd year)

Projects

• VIYGAN-2: multi-date GAN super-resolution of Sentinel-2 for India, 10 m to 2.39 m (SIH26142, NTRO)
• rail-controller: CP-SAT train precedence and crossing decisions on any Indian Railways corridor
• KalJahaz: dry-bulk freight cost forecasting for India's East Coast ports (Smart India Hackathon)
• ml-preprocessing: data cleaning and normalisation pipeline for any ML model (denoising, NaN-safe handling, translation / scale / rotation normalisation, feature engineering, unit-tested invariances); demo: 44 hand signs to speech at 2.15 ms per prediction on a Raspberry Pi
• alfred: offline voice assistant on a local 9B model, personality by activation steering
• NIP: low-latency PC to Raspberry Pi screen streaming in C
• always-thinking mind: a never-prompted local LLM with a memory graph (ongoing research)
• hardware: ESP32-CAM streaming with YOLO detection, GPS, a Bluetooth car

Full version

resume.pdf

one page, ready to download

The fastest way to reach me is email.

github

https://github.com/Sarkar-Aveek

email

aveeksarkar.official@gmail.com

Or type: open email