PhatPipe

A concurrent file downloader project to demonstrate network programming using goroutines, streaming IO, file writing, and progress tracking.

Features

• Concurrent Chunked Downloading: Splits files into multiple chunks using HTTP Range requests and downloads them in parallel.
• Multi-Bar Progress UI: Real-time progress tracking for each individual chunk with speed and ETA calculations.
• Graceful Fallback: Automatically reverts to a single-stream download if the server doesn't support range requests.
• Built-in Test Server: Includes a local HTTP server to simulate high-concurrency download scenarios.
• Path Protection: Built-in security checks to prevent path traversal and ensure directory writability.

Installation

# Clone the repository
git clone https://github.com/yourusername/go-phatpipe.git
cd go-phatpipe

# Build the binary
go build -o phatpipe main.go test_server.go

Usage

Run the Downloader

./phatpipe -url https://example.com/largefile.zip -chunks 8

Run the Test Server

To experiment locally, you can start the included test server:

# Terminal 1: Start the server
./phatpipe -serve -port 8080

# Terminal 2: Download from the local server
./phatpipe -url http://localhost:8080/testdata -chunks 6

CLI Flags

Flag	Default	Description
-url	(Required)	The URL of the file to download.
-chunks	4	Number of concurrent chunks to download.
-retries	3	Number of retries for failed chunks.
-dir	./	Directory to save the downloaded file.
-filename	(auto)	Custom filename (defaults to URL basename).
-serve	false	Enable test server mode.
-port	8080	Port for the test server.

Technical Details

• Concurrency: Uses sync.WaitGroup to coordinate chunk downloads and an error channel for failure handling.
• Atomic Operations: Uses sync/atomic for thread-safe progress tracking across multiple goroutines.
• Streaming IO: Uses io.Copy with a custom ProgressWriter to pipe data directly to disk, minimizing memory overhead.
• Chunk Management: Manages temporary .partN files and performs an ordered merge once all chunks complete.