Loading...

Media is loading
 

Degree

Bachelor of Science (Computer Science)

Department

Department of Computer Science

School

School of Mathematics and Computer Science (SMCS)

Advisor

Ms. Tasbiha Fatima, Lecturer, Department of Computer Science

Co-Advisor

Abdul Wahab, Visiting Faculty (Computer Sciences)

Keywords

Computer Vision, Geospatial Analysis, MLOps, Generative AI, Deep Learning

Abstract

Roadlytics is a road-quality assessment platform that combines deep learning over Sentinel-2 satellite imagery with a deployed web application for analysts working on rural infrastructure in Sindh, Pakistan. The analytical pipeline operates in two stages. The first performs binary semantic segmentation of road pixels using a DeepLabV3+ architecture with an SE-ResNeXt-101-32x4d encoder; through six sequential experiments, test IoU was raised from a 31.4% baseline to 66.2%, satisfying the project’s target band of 65–75%. The single most consequential intervention was a training-time 3×3 morphological dilation of one-pixel road labels, which addressed a topological degeneracy in the ground truth more effectively than shallower encoders or alternative decoder families such as U-Net. The second stage classifies each road pixel as Good, Unpaved, or Damaged using an EfficientNet-B2 trained on K-Means pseudo-labels derived from spectral features, achieving 83% accuracy and a 0.83 macro F1 on held-out test patches. The pipeline is delivered as a production web application in which analysts upload georeferenced imagery, run jobs on GPU-backed inference containers, and inspect results as interactive map overlays alongside connectivity metrics and downloadable reports. A retrieval-augmented assistant, built on a persistent vector database with deterministic local SHA-1 hash embeddings and a Gemini generation backend, sits over job metadata, artifact manifests, analytics snapshots, and report text to produce grounded summaries, failure diagnostics, and report drafts while explicitly refraining from unsupported claims about field truth, pavement safety, or repair cost.

Tools and Technologies Used

The platform is implemented as a Python and TypeScript codebase spanning a browser client and a backend service. The frontend uses Next.js 15 (App Router), React 19, TypeScript, Leaflet, and react-leaflet, and is built as a static export served by nginx. The backend is implemented in FastAPI with Uvicorn, running on Python 3.11, with Pydantic, an in-process async worker queue, and SQLite (Python standard-library sqlite3) for job, artifact, event, and analytics persistence. The application has no authentication layer in the current no-auth MVP. Object storage is handled by Azure Blob Storage via the azurestorage-blob SDK, with a local filesystem fallback when no Azure connection string is configured. The inference pipeline runs in the same container as the backend and uses PyTorch, segmentation_models_pytorch, rasterio, GDAL, NumPy, OpenCV, scikit-learn, albumentations, scipy, geopandas, shapely, pyproj, fiona, and networkx. The deep-learning models are DeepLabV3+ with an SE-ResNeXt-101-32x4d encoder for road segmentation, and EfficientNet-B2 (torchvision) modified for four-channel input for condition classification;both were trained on four-band Sentinel-2 Level-2 GeoTIFF tiles covering the Blue, Green, Red, and Near-Infrared bands. A PakOSM segmentation alternative rasterizes Pakistan OpenStreetMap road geometries onto the GeoTIFF grid without requiring learned weights. Connectivity analytics are computed with networkx. The retrieval-augmented assistant is built on ChromaDB for vector storage, a deterministic local SHA-1 hash embedding function, and the Google Gemini API accessed through the google-genai SDK (model: gemini-2.5flash), with an extractive fallback path when no API key is configured. The map view renders user-toggleable raster overlays and GeoJSON layers above a fixed OpenStreetMap basemap via Leaflet, with tile endpoints served by rio-tiler. Supporting tooling includes npm, ESLint, TypeScript, ruff, mypy, pytest with pytest-asyncio, and Docker Compose for orchestration. The system is deployed as two Docker containers: the backend (Python/FastAPI/Uvicorn) and the frontend (Next.js static export behind nginx).

Methodology

Development proceeded along two parallel tracks. The machine-learning track adopted sequential ablation as its core methodology: beginning with a U-Net/ResNet-34 baseline at 31.4% test IoU, each subsequent experiment isolated a single variable — such as batch size, label representation, encoder depth, or decoder family — so that observed improvements could be attributed unambiguously to that change. The severe foreground–background imbalance characteristic of road segmentation, in which road pixels constitute under two percent of the image, was addressed through stratified tile filtering, a combined Focal–Dice loss, and a reduced prediction threshold. The most significant intervention was a 3×3 morphological dilation applied to the one-pixel road labels within the data loader, which provided the network with a topology-preserving learning target. The final deployed segmentation model is DeepLabV3+ with an SE-ResNeXt-101-32x4d encoder, which superseded the U-Net experiments and runs sliding-window inference over 1024×1024 tiles with a 512-pixel stride. A PakOSM alternative directly rasterizes Pakistan OpenStreetMap road geometries onto the input GeoTIFF grid as a no-weights segmentation option. For road-condition assessment, an unsupervised-to-supervised strategy was adopted: K-Means clustering over spectral features produced pseudo-labels at scale, and an EfficientNet-B2 modified for four-channel input was trained on 32×32 patches sampled around road pixels with class-balanced batching. The software-engineering track followed a contract-driven methodology in which the system was decomposed into self-contained modules, each with explicit dependencies, file boundaries, and acceptance criteria, enabling parallel development without breaking inter-module interfaces. The runtime architecture is a FastAPI backend with an in-process async worker queue; job submissions are placed on the queue, progress is persisted in SQLite, and the browser polls the /jobs/{id} endpoint for status updates. When Azure Blob Storage is configured, client uploads proceed directly to object storage via SAS tokens, bypassing the application server; otherwise, uploads proxythrough the backend to local storage.

Document Type

Restricted Access

Submission Type

BSCS Final Year Project

Download

Share

COinS