Node.js Streams

Lesson content

🎯 Goal By the end of this lesson, you'll understand: What are Streams? Why do we use Streams? Types of Streams How Streams work Real-world examples Interview questions What are Streams? A Stream is a way to read or write data continuously in small pieces (chunks) instead of loading the entire data into memory. Simple Definition: Streams process data chunk by chunk , making them fast and memory-efficient. Why Do We Need Streams? Imagine reading a 10 GB file . Without Streams: ❌ Load the entire file into memory. With Streams: ✅ Read small chunks one by one. This saves memory and improves performance. Real-Life Example 🚰 Think of a water pipe . Water Tank │ ▼ Pipe │ ▼ Bucket Water flows continuously. Similarly, File → Data Source Stream → Pipe Application → Bucket Types of Streams Node.js has 4 types of Streams . Stream Purpose Readable Read data Writable Write data Duplex Read and Write Transform Read, Modify, and Write 1. Readable Stream Used to read data. Example const fs = require ( "fs" ); const readStream = fs . createReadStream( "input.txt" ); readStream . on( "data" , ( chunk ) => { console . log( chunk . toString()); }); Output (File content printed chunk by chunk) 2. Writable Stream Used to write data. Example const fs = require ( "fs" ); const writeStream = fs . createWriteStream( "output.txt" ); writeStream . write( "Hello Node.js" ); writeStream . end(); 3. Duplex Stream Can read and write . Examples: TCP Socket Network Connection Read ▲ │ Duplex │ ▼ Write 4. Transform Stream Reads data, modifies it, then writes it. Examples: Compression Encryption Decompression Input │ ▼ Transform │ ▼ Output How Streams Work Large File │ ▼ Read Chunk │ ▼ Process Chunk │ ▼ Next Chunk │ ▼ Finished Example const fs = require ( "fs" ); const stream = fs . createReadStream( "input.txt" ); stream . on( "data" , ( chunk ) => { console . log( chunk . toString()); }); stream . on( "end" , () => { console . log( "Reading Complete" ); }); Output Chunk 1 Chunk 2 Chunk 3 Reading Complete Pipe() pipe() connects a readable stream to a writable stream. Example const fs = require ( "fs" ); const read = fs . createReadStream( "input.txt" ); const write = fs . createWriteStream( "copy.txt" ); read . pipe( write ); input.txt │ ▼ Readable Stream │ ▼ pipe() │ ▼ Writable Stream │ ▼ copy.txt Stream Events Event Description data New chunk received end Reading finished error Error occurred finish Writing finished close Stream closed Streams vs readFile() Feature Streams fs.readFile() Memory Usage Low High Reads Data Chunk by chunk Entire file Large Files ✅ Best ❌ Not ideal Performance Faster for large files Slower for very large files Real-Life Example 🎥 Watching a movie online. Movie Server │ ▼ Video Stream │ ▼ Your Device The whole movie is not downloaded first . It is streamed continuously. Where Are Streams Used? ✅ Video Streaming ✅ Audio Streaming ✅ File Uploads ✅ File Downloads ✅ Large File Processing ✅ Data Compression ✅ Network Communication Interview Question Which stream type can both read and write? Answer: Duplex Stream Which stream modifies data while passing it? Answer: Transform Stream Best Practices ✅ Use Streams for large files. ✅ Use pipe() to efficiently transfer data between streams. ✅ Handle error events to avoid application crashes. ❌ Avoid fs.readFile() for very large files. Key Points ✅ Streams process data in small chunks. ✅ Streams are memory-efficient. ✅ There are four stream types: Readable, Writable, Duplex, and Transform. ✅ pipe() connects readable and writable streams. ✅ Streams are ideal for large files and real-time data. 🧠 Easy Formula Large File │ ▼ Readable Stream │ ▼ Process Data │ ▼ Writable Stream Readable → Read Writable → Write Duplex → Read + Write Transform → Read + Modify + Write Remember: 🌊 Streams process data chunk by chunk. 💾 They reduce memory usage and improve performance. 🔄 Use pipe() to connect streams efficiently. 🚀 Streams are the preferred choice for handling large files and real-time data in Node.js.