
Nodejs Development
- 13 installs
- 1 repo stars
- Updated November 29, 2025
- manutej/crush-mcp-server
Helps with ai & agent building tasks.
About
nodejs-development is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted development.
- nodejs-development
- AI & Agent Building
- AI-coding skill
Nodejs Development by the numbers
- 13 all-time installs (skills.sh)
- Ranked #11,409 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 2, 2026 (Skillselion catalog sync)
npx skills add https://github.com/manutej/crush-mcp-server --skill nodejs-developmentAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 13 |
|---|---|
| repo stars | ★ 1 |
| Last updated | November 29, 2025 |
| Repository | manutej/crush-mcp-server ↗ |
What it does
Helps with ai & agent building tasks.
Files
Node.js Development Skill
This skill provides comprehensive guidance for building modern Node.js applications covering the event loop, async patterns, streams, file system operations, HTTP servers, process management, and best practices for production-ready Node.js development.
When to Use This Skill
Use this skill when:
- Building RESTful APIs and backend services
- Creating command-line interface (CLI) tools and utilities
- Developing microservices and distributed systems
- Building real-time applications with WebSockets or Server-Sent Events
- Creating build tools, task runners, and development tools
- Working with file processing and data transformation
- Implementing server-side data validation and business logic
- Building proxy servers and middleware layers
- Creating automation scripts and system utilities
- Developing serverless functions and cloud-native applications
- Implementing background job processors and workers
- Building GraphQL servers and API gateways
Core Concepts
Event Loop
The event loop is the heart of Node.js, enabling non-blocking I/O operations despite JavaScript being single-threaded.
Event Loop Phases:
// The event loop processes operations in this order:
// 1. Timers (setTimeout, setInterval)
// 2. Pending callbacks (I/O callbacks deferred from previous iteration)
// 3. Idle, prepare (internal use)
// 4. Poll (retrieve new I/O events)
// 5. Check (setImmediate callbacks)
// 6. Close callbacks (socket.on('close'))
// Understanding execution order
console.log('1 - Start');
setTimeout(() => {
console.log('2 - Timeout');
}, 0);
setImmediate(() => {
console.log('3 - Immediate');
});
Promise.resolve().then(() => {
console.log('4 - Promise');
});
process.nextTick(() => {
console.log('5 - Next Tick');
});
console.log('6 - End');
// Output order: 1, 6, 5, 4, 2, 3
// (process.nextTick and Promises run before other phases)Event Loop Best Practices:
// Don't block the event loop
// Bad - blocking operation
const data = fs.readFileSync('large-file.txt'); // Blocks
// Good - non-blocking
fs.readFile('large-file.txt', (err, data) => {
// Non-blocking
});
// Better - using promises
const data = await fs.promises.readFile('large-file.txt');
// Avoid heavy CPU operations
// Bad - blocks event loop
function fibonacci(n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}
// Good - offload to worker threads
const { Worker } = require('worker_threads');
function computeInWorker(n) {
return new Promise((resolve, reject) => {
const worker = new Worker('./fibonacci-worker.js', {
workerData: n
});
worker.on('message', resolve);
worker.on('error', reject);
});
}Modules
Node.js supports both CommonJS and ES Modules for organizing code.
CommonJS (Traditional):
// math.js - Exporting
function add(a, b) {
return a + b;
}
function subtract(a, b) {
return a - b;
}
module.exports = { add, subtract };
// Alternative export syntax
exports.multiply = (a, b) => a * b;
// app.js - Importing
const { add, subtract } = require('./math');
const fs = require('fs'); // Built-in module
const express = require('express'); // npm package
console.log(add(5, 3)); // 8ES Modules (Modern):
// math.mjs or math.js (with "type": "module" in package.json)
export function add(a, b) {
return a + b;
}
export function subtract(a, b) {
return a - b;
}
export default class Calculator {
add(a, b) { return a + b; }
}
// app.mjs - Importing
import { add, subtract } from './math.mjs';
import Calculator from './math.mjs';
import fs from 'fs';
import express from 'express';
console.log(add(5, 3)); // 8Built-in Modules:
// Core Node.js modules (no npm install required)
const fs = require('fs'); // File system
const path = require('path'); // Path utilities
const http = require('http'); // HTTP server
const https = require('https'); // HTTPS server
const crypto = require('crypto'); // Cryptography
const os = require('os'); // Operating system
const events = require('events'); // Event emitter
const stream = require('stream'); // Streams
const util = require('util'); // Utilities
const child_process = require('child_process'); // Child processes
const cluster = require('cluster'); // Clustering
const url = require('url'); // URL parsing
const querystring = require('querystring'); // Query stringsAsync Programming
Node.js provides multiple patterns for handling asynchronous operations.
Callbacks (Traditional):
const fs = require('fs');
// Error-first callback pattern
fs.readFile('file.txt', 'utf8', (err, data) => {
if (err) {
console.error('Error reading file:', err);
return;
}
console.log('File contents:', data);
});
// Callback hell (avoid this)
fs.readFile('file1.txt', (err, data1) => {
if (err) return console.error(err);
fs.readFile('file2.txt', (err, data2) => {
if (err) return console.error(err);
fs.readFile('file3.txt', (err, data3) => {
if (err) return console.error(err);
console.log(data1, data2, data3);
});
});
});Promises:
const fs = require('fs').promises;
// Basic promise
fs.readFile('file.txt', 'utf8')
.then(data => {
console.log('File contents:', data);
return data;
})
.catch(err => {
console.error('Error:', err);
})
.finally(() => {
console.log('Operation complete');
});
// Promise chaining
fs.readFile('file1.txt', 'utf8')
.then(data1 => {
console.log('File 1:', data1);
return fs.readFile('file2.txt', 'utf8');
})
.then(data2 => {
console.log('File 2:', data2);
return fs.readFile('file3.txt', 'utf8');
})
.then(data3 => {
console.log('File 3:', data3);
})
.catch(err => {
console.error('Error:', err);
});
// Promise.all for parallel operations
Promise.all([
fs.readFile('file1.txt', 'utf8'),
fs.readFile('file2.txt', 'utf8'),
fs.readFile('file3.txt', 'utf8')
])
.then(([data1, data2, data3]) => {
console.log(data1, data2, data3);
})
.catch(err => {
console.error('Error:', err);
});
// Promise utilities
Promise.race([
fetch('https://api1.com'),
fetch('https://api2.com')
]); // Returns first to complete
Promise.allSettled([
promise1,
promise2,
promise3
]); // Waits for all, returns all results (success or failure)
Promise.any([
promise1,
promise2
]); // Returns first successful promiseAsync/Await (Modern):
const fs = require('fs').promises;
// Basic async/await
async function readFile() {
try {
const data = await fs.readFile('file.txt', 'utf8');
console.log('File contents:', data);
return data;
} catch (err) {
console.error('Error:', err);
throw err;
}
}
// Sequential execution
async function readFilesSequentially() {
try {
const data1 = await fs.readFile('file1.txt', 'utf8');
const data2 = await fs.readFile('file2.txt', 'utf8');
const data3 = await fs.readFile('file3.txt', 'utf8');
return [data1, data2, data3];
} catch (err) {
console.error('Error:', err);
throw err;
}
}
// Parallel execution
async function readFilesParallel() {
try {
const [data1, data2, data3] = await Promise.all([
fs.readFile('file1.txt', 'utf8'),
fs.readFile('file2.txt', 'utf8'),
fs.readFile('file3.txt', 'utf8')
]);
return [data1, data2, data3];
} catch (err) {
console.error('Error:', err);
throw err;
}
}
// Top-level await (ES modules only)
const data = await fs.readFile('config.json', 'utf8');
const config = JSON.parse(data);
// Error handling patterns
async function robustOperation() {
try {
const result = await riskyOperation();
return { success: true, data: result };
} catch (err) {
console.error('Operation failed:', err);
return { success: false, error: err.message };
}
}
// Multiple error handling
async function multipleOperations() {
const results = await Promise.allSettled([
operation1(),
operation2(),
operation3()
]);
results.forEach((result, index) => {
if (result.status === 'fulfilled') {
console.log(`Operation ${index} succeeded:`, result.value);
} else {
console.error(`Operation ${index} failed:`, result.reason);
}
});
}Promisify Utility:
const util = require('util');
const fs = require('fs');
// Convert callback-based function to promise-based
const readFile = util.promisify(fs.readFile);
const writeFile = util.promisify(fs.writeFile);
async function processFile() {
const data = await readFile('input.txt', 'utf8');
const processed = data.toUpperCase();
await writeFile('output.txt', processed);
}
// Custom promisify
function promisify(fn) {
return (...args) => {
return new Promise((resolve, reject) => {
fn(...args, (err, result) => {
if (err) reject(err);
else resolve(result);
});
});
};
}Streams
Streams are one of Node.js's most powerful features for handling data efficiently.
Stream Types:
const fs = require('fs');
const { Readable, Writable, Duplex, Transform } = require('stream');
// 1. Readable Stream
const readStream = fs.createReadStream('large-file.txt', {
encoding: 'utf8',
highWaterMark: 64 * 1024 // 64KB chunks
});
readStream.on('data', (chunk) => {
console.log('Received chunk:', chunk.length);
});
readStream.on('end', () => {
console.log('Finished reading');
});
readStream.on('error', (err) => {
console.error('Error:', err);
});
// 2. Writable Stream
const writeStream = fs.createWriteStream('output.txt');
writeStream.write('Hello ');
writeStream.write('World\n');
writeStream.end();
writeStream.on('finish', () => {
console.log('Finished writing');
});
// 3. Pipe - connecting streams
fs.createReadStream('input.txt')
.pipe(fs.createWriteStream('output.txt'));
// 4. Transform Stream
class UpperCaseTransform extends Transform {
_transform(chunk, encoding, callback) {
this.push(chunk.toString().toUpperCase());
callback();
}
}
fs.createReadStream('input.txt')
.pipe(new UpperCaseTransform())
.pipe(fs.createWriteStream('output.txt'));
// 5. Custom Readable Stream
class NumberStream extends Readable {
constructor(max) {
super();
this.current = 0;
this.max = max;
}
_read() {
if (this.current <= this.max) {
this.push(String(this.current++));
} else {
this.push(null); // End stream
}
}
}
const numberStream = new NumberStream(10);
numberStream.on('data', (num) => {
console.log(num);
});
// 6. Custom Writable Stream
class LogStream extends Writable {
_write(chunk, encoding, callback) {
console.log(`[LOG] ${chunk.toString()}`);
callback();
}
}
const logStream = new LogStream();
logStream.write('Message 1\n');
logStream.write('Message 2\n');Stream Patterns:
const { pipeline } = require('stream');
const { createReadStream, createWriteStream } = require('fs');
const { createGzip } = require('zlib');
// Pipeline - handles errors and cleanup automatically
pipeline(
createReadStream('input.txt'),
createGzip(),
createWriteStream('input.txt.gz'),
(err) => {
if (err) {
console.error('Pipeline failed:', err);
} else {
console.log('Pipeline succeeded');
}
}
);
// Stream utilities
const { finished, pipeline: promisePipeline } = require('stream/promises');
async function processFile() {
await promisePipeline(
createReadStream('input.txt'),
createGzip(),
createWriteStream('output.txt.gz')
);
console.log('Processing complete');
}
// Backpressure handling
const reader = createReadStream('large-file.txt');
const writer = createWriteStream('output.txt');
reader.on('data', (chunk) => {
const canContinue = writer.write(chunk);
if (!canContinue) {
// Pause reading if write buffer is full
reader.pause();
}
});
writer.on('drain', () => {
// Resume reading when write buffer is drained
reader.resume();
});File System
The fs module provides file system operations.
Reading Files:
const fs = require('fs');
const fsPromises = require('fs').promises;
// Synchronous (blocking - avoid in production)
try {
const data = fs.readFileSync('file.txt', 'utf8');
console.log(data);
} catch (err) {
console.error(err);
}
// Callback-based
fs.readFile('file.txt', 'utf8', (err, data) => {
if (err) {
console.error('Error reading file:', err);
return;
}
console.log(data);
});
// Promise-based
fsPromises.readFile('file.txt', 'utf8')
.then(data => console.log(data))
.catch(err => console.error(err));
// Async/await
async function readFile() {
try {
const data = await fsPromises.readFile('file.txt', 'utf8');
console.log(data);
} catch (err) {
console.error('Error:', err);
}
}
// Read JSON file
async function readJSON(filename) {
const data = await fsPromises.readFile(filename, 'utf8');
return JSON.parse(data);
}Writing Files:
// Write file (overwrites if exists)
await fsPromises.writeFile('output.txt', 'Hello World');
// Append to file
await fsPromises.appendFile('log.txt', 'New log entry\n');
// Write JSON
async function writeJSON(filename, data) {
await fsPromises.writeFile(
filename,
JSON.stringify(data, null, 2)
);
}
// Atomic write (write to temp, then rename)
async function atomicWrite(filename, data) {
const tempFile = `${filename}.tmp`;
await fsPromises.writeFile(tempFile, data);
await fsPromises.rename(tempFile, filename);
}File Operations:
const path = require('path');
// Check if file exists
async function fileExists(filename) {
try {
await fsPromises.access(filename);
return true;
} catch {
return false;
}
}
// Get file stats
const stats = await fsPromises.stat('file.txt');
console.log({
size: stats.size,
isFile: stats.isFile(),
isDirectory: stats.isDirectory(),
modified: stats.mtime,
created: stats.birthtime
});
// Copy file
await fsPromises.copyFile('source.txt', 'dest.txt');
// Move/rename file
await fsPromises.rename('old-name.txt', 'new-name.txt');
// Delete file
await fsPromises.unlink('file-to-delete.txt');
// Create directory
await fsPromises.mkdir('new-directory', { recursive: true });
// Read directory
const files = await fsPromises.readdir('directory');
console.log(files);
// Read directory with file types
const entries = await fsPromises.readdir('directory', {
withFileTypes: true
});
for (const entry of entries) {
if (entry.isFile()) {
console.log('File:', entry.name);
} else if (entry.isDirectory()) {
console.log('Directory:', entry.name);
}
}
// Remove directory
await fsPromises.rmdir('directory');
// Remove directory recursively
await fsPromises.rm('directory', { recursive: true, force: true });
// Watch for file changes
const watcher = fs.watch('file.txt', (eventType, filename) => {
console.log(`File ${filename} changed: ${eventType}`);
});
// Stop watching
watcher.close();Path Utilities:
const path = require('path');
// Join paths (handles separators correctly)
const fullPath = path.join('/users', 'john', 'documents', 'file.txt');
// /users/john/documents/file.txt
// Resolve absolute path
const absolute = path.resolve('documents', 'file.txt');
// /current/working/directory/documents/file.txt
// Get directory name
path.dirname('/users/john/file.txt'); // /users/john
// Get file name
path.basename('/users/john/file.txt'); // file.txt
path.basename('/users/john/file.txt', '.txt'); // file
// Get extension
path.extname('file.txt'); // .txt
// Parse path
const parsed = path.parse('/users/john/file.txt');
// {
// root: '/',
// dir: '/users/john',
// base: 'file.txt',
// ext: '.txt',
// name: 'file'
// }
// Format path
const formatted = path.format({
dir: '/users/john',
base: 'file.txt'
}); // /users/john/file.txt
// Normalize path
path.normalize('/users/john/../jane/./file.txt');
// /users/jane/file.txt
// Check if absolute
path.isAbsolute('/users/john'); // true
path.isAbsolute('documents/file.txt'); // false
// Get relative path
path.relative('/users/john', '/users/jane/file.txt');
// ../jane/file.txtHTTP/HTTPS
Creating HTTP servers and making HTTP requests.
HTTP Server:
const http = require('http');
// Basic HTTP server
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('Hello World\n');
});
server.listen(3000, () => {
console.log('Server running at http://localhost:3000/');
});
// Server with routing
const server = http.createServer((req, res) => {
const { method, url } = req;
if (url === '/' && method === 'GET') {
res.writeHead(200, { 'Content-Type': 'text/html' });
res.end('<h1>Home Page</h1>');
} else if (url === '/api/users' && method === 'GET') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ users: ['Alice', 'Bob'] }));
} else if (url === '/api/users' && method === 'POST') {
let body = '';
req.on('data', chunk => {
body += chunk.toString();
});
req.on('end', () => {
const data = JSON.parse(body);
res.writeHead(201, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ success: true, data }));
});
} else {
res.writeHead(404, { 'Content-Type': 'text/plain' });
res.end('Not Found');
}
});
server.listen(3000);
// JSON API server
const server = http.createServer(async (req, res) => {
try {
if (req.url === '/api/data' && req.method === 'GET') {
const data = { message: 'Hello', timestamp: Date.now() };
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(data));
} else if (req.url === '/api/data' && req.method === 'POST') {
const body = await getRequestBody(req);
const data = JSON.parse(body);
// Process data
const result = { received: data, processed: true };
res.writeHead(201, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(result));
} else {
res.writeHead(404);
res.end(JSON.stringify({ error: 'Not found' }));
}
} catch (err) {
res.writeHead(500);
res.end(JSON.stringify({ error: err.message }));
}
});
function getRequestBody(req) {
return new Promise((resolve, reject) => {
let body = '';
req.on('data', chunk => {
body += chunk.toString();
});
req.on('end', () => resolve(body));
req.on('error', reject);
});
}HTTPS Server:
const https = require('https');
const fs = require('fs');
const options = {
key: fs.readFileSync('private-key.pem'),
cert: fs.readFileSync('certificate.pem')
};
const server = https.createServer(options, (req, res) => {
res.writeHead(200);
res.end('Secure Hello World\n');
});
server.listen(443);Making HTTP Requests:
const https = require('https');
// Basic GET request
https.get('https://api.example.com/data', (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
console.log(JSON.parse(data));
});
}).on('error', (err) => {
console.error('Error:', err.message);
});
// POST request
const postData = JSON.stringify({ name: 'John', age: 30 });
const options = {
hostname: 'api.example.com',
port: 443,
path: '/users',
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(postData)
}
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
console.log('Response:', JSON.parse(data));
});
});
req.on('error', (err) => {
console.error('Error:', err.message);
});
req.write(postData);
req.end();
// Promise wrapper for HTTP requests
function httpRequest(url, options = {}) {
return new Promise((resolve, reject) => {
const req = https.request(url, options, (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
resolve({
statusCode: res.statusCode,
headers: res.headers,
body: data
});
});
});
req.on('error', reject);
if (options.body) {
req.write(options.body);
}
req.end();
});
}
// Using fetch (Node.js 18+)
async function fetchData() {
const response = await fetch('https://api.example.com/data');
const data = await response.json();
console.log(data);
}
async function postData() {
const response = await fetch('https://api.example.com/users', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ name: 'John', age: 30 })
});
const result = await response.json();
console.log(result);
}Process and Environment
Managing process lifecycle and environment variables.
Process Object:
// Process information
console.log('Node version:', process.version);
console.log('Platform:', process.platform);
console.log('Architecture:', process.arch);
console.log('Process ID:', process.pid);
console.log('Current directory:', process.cwd());
console.log('Memory usage:', process.memoryUsage());
console.log('CPU usage:', process.cpuUsage());
console.log('Uptime:', process.uptime());
// Command-line arguments
// node app.js --port 3000 --host localhost
console.log('Arguments:', process.argv);
// ['node', '/path/to/app.js', '--port', '3000', '--host', 'localhost']
// Parse arguments
function parseArgs() {
const args = {};
for (let i = 2; i < process.argv.length; i += 2) {
const key = process.argv[i].replace('--', '');
const value = process.argv[i + 1];
args[key] = value;
}
return args;
}
const config = parseArgs();
console.log(config); // { port: '3000', host: 'localhost' }
// Environment variables
const port = process.env.PORT || 3000;
const nodeEnv = process.env.NODE_ENV || 'development';
const dbUrl = process.env.DATABASE_URL;
console.log('Port:', port);
console.log('Environment:', nodeEnv);
// Set environment variable
process.env.MY_VAR = 'value';
// Exit process
process.exit(0); // Success
process.exit(1); // Error
// Exit codes
const EXIT_CODES = {
SUCCESS: 0,
GENERAL_ERROR: 1,
INVALID_ARGUMENT: 2,
CONFIG_ERROR: 3
};
if (!config.isValid) {
console.error('Invalid configuration');
process.exit(EXIT_CODES.CONFIG_ERROR);
}
// Process events
process.on('exit', (code) => {
console.log(`Process exiting with code ${code}`);
});
process.on('uncaughtException', (err) => {
console.error('Uncaught exception:', err);
process.exit(1);
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled rejection:', reason);
process.exit(1);
});
process.on('SIGTERM', () => {
console.log('SIGTERM received, shutting down gracefully');
server.close(() => {
process.exit(0);
});
});
process.on('SIGINT', () => {
console.log('SIGINT received, shutting down gracefully');
process.exit(0);
});
// Send signals to process
process.kill(process.pid, 'SIGTERM');Child Processes:
const { exec, execFile, spawn, fork } = require('child_process');
// exec - run shell command
exec('ls -la', (err, stdout, stderr) => {
if (err) {
console.error('Error:', err);
return;
}
console.log('Output:', stdout);
console.error('Errors:', stderr);
});
// execFile - run executable directly (safer)
execFile('node', ['--version'], (err, stdout, stderr) => {
if (err) {
console.error('Error:', err);
return;
}
console.log('Node version:', stdout);
});
// spawn - for long-running processes or large output
const ls = spawn('ls', ['-la', '/usr']);
ls.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
ls.stderr.on('data', (data) => {
console.error(`stderr: ${data}`);
});
ls.on('close', (code) => {
console.log(`Process exited with code ${code}`);
});
// fork - spawn Node.js processes
// parent.js
const child = fork('./child.js');
child.on('message', (msg) => {
console.log('Message from child:', msg);
});
child.send({ hello: 'child' });
// child.js
process.on('message', (msg) => {
console.log('Message from parent:', msg);
});
process.send({ hello: 'parent' });
// Promise-based exec
const util = require('util');
const execPromise = util.promisify(exec);
async function runCommand() {
try {
const { stdout, stderr } = await execPromise('ls -la');
console.log('Output:', stdout);
} catch (err) {
console.error('Error:', err);
}
}Cluster Module:
const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master process ${process.pid} is running`);
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`Worker ${worker.process.pid} died`);
// Restart worker
cluster.fork();
});
} else {
// Workers share the same port
http.createServer((req, res) => {
res.writeHead(200);
res.end(`Process ${process.pid} handled request\n`);
}).listen(8000);
console.log(`Worker ${process.pid} started`);
}API Reference
Buffer
Buffers handle binary data in Node.js.
// Create buffers
const buf1 = Buffer.from('Hello World');
const buf2 = Buffer.from([72, 101, 108, 108, 111]);
const buf3 = Buffer.alloc(10); // 10 bytes, filled with 0
const buf4 = Buffer.allocUnsafe(10); // Faster, but may contain old data
// Read/write buffers
buf3.write('Hello');
console.log(buf3.toString()); // Hello
console.log(buf3.toString('hex')); // Hex representation
// Buffer operations
const buf5 = Buffer.concat([buf1, buf2]);
const buf6 = buf1.slice(0, 5); // Hello
// Buffer comparison
Buffer.compare(buf1, buf2); // -1, 0, or 1
// JSON conversion
const json = JSON.stringify(buf1);
const parsed = Buffer.from(JSON.parse(json));Events
EventEmitter is the foundation of Node.js's event-driven architecture.
const EventEmitter = require('events');
// Create event emitter
class MyEmitter extends EventEmitter {}
const myEmitter = new MyEmitter();
// Listen for events
myEmitter.on('event', (arg1, arg2) => {
console.log('Event occurred:', arg1, arg2);
});
// Emit events
myEmitter.emit('event', 'arg1', 'arg2');
// Listen once
myEmitter.once('oneTime', () => {
console.log('This will only fire once');
});
// Remove listener
function handler() {
console.log('Handler called');
}
myEmitter.on('event', handler);
myEmitter.off('event', handler);
// Error handling
myEmitter.on('error', (err) => {
console.error('Error occurred:', err);
});
// Real-world example: Custom logger
class Logger extends EventEmitter {
log(message) {
this.emit('log', { message, timestamp: Date.now() });
}
error(message) {
this.emit('error', { message, timestamp: Date.now() });
}
}
const logger = new Logger();
logger.on('log', (data) => {
console.log(`[${new Date(data.timestamp).toISOString()}] ${data.message}`);
});
logger.on('error', (data) => {
console.error(`[${new Date(data.timestamp).toISOString()}] ERROR: ${data.message}`);
});
logger.log('Application started');
logger.error('Something went wrong');Crypto
Cryptographic operations for security.
const crypto = require('crypto');
// Generate random data
const randomBytes = crypto.randomBytes(16).toString('hex');
console.log('Random:', randomBytes);
// Hash data (one-way)
const hash = crypto.createHash('sha256');
hash.update('password123');
console.log('Hash:', hash.digest('hex'));
// HMAC (keyed hash)
const hmac = crypto.createHmac('sha256', 'secret-key');
hmac.update('message');
console.log('HMAC:', hmac.digest('hex'));
// Password hashing with salt
function hashPassword(password) {
const salt = crypto.randomBytes(16).toString('hex');
const hash = crypto.pbkdf2Sync(password, salt, 100000, 64, 'sha512').toString('hex');
return { salt, hash };
}
function verifyPassword(password, salt, hash) {
const hashToVerify = crypto.pbkdf2Sync(password, salt, 100000, 64, 'sha512').toString('hex');
return hash === hashToVerify;
}
const { salt, hash } = hashPassword('mypassword');
console.log(verifyPassword('mypassword', salt, hash)); // true
// Encryption/Decryption
function encrypt(text, key) {
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv('aes-256-cbc', Buffer.from(key), iv);
let encrypted = cipher.update(text);
encrypted = Buffer.concat([encrypted, cipher.final()]);
return iv.toString('hex') + ':' + encrypted.toString('hex');
}
function decrypt(text, key) {
const parts = text.split(':');
const iv = Buffer.from(parts[0], 'hex');
const encrypted = Buffer.from(parts[1], 'hex');
const decipher = crypto.createDecipheriv('aes-256-cbc', Buffer.from(key), iv);
let decrypted = decipher.update(encrypted);
decrypted = Buffer.concat([decrypted, decipher.final()]);
return decrypted.toString();
}
const key = crypto.randomBytes(32); // 256 bits
const encrypted = encrypt('Hello World', key);
const decrypted = decrypt(encrypted, key);
console.log(decrypted); // Hello WorldWorkflow Patterns
REST API Server with Express
const express = require('express');
const app = express();
// Middleware
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
// Logging middleware
app.use((req, res, next) => {
console.log(`${req.method} ${req.url}`);
next();
});
// Routes
app.get('/api/users', async (req, res) => {
try {
const users = await db.getUsers();
res.json(users);
} catch (err) {
res.status(500).json({ error: err.message });
}
});
app.get('/api/users/:id', async (req, res) => {
try {
const user = await db.getUserById(req.params.id);
if (!user) {
return res.status(404).json({ error: 'User not found' });
}
res.json(user);
} catch (err) {
res.status(500).json({ error: err.message });
}
});
app.post('/api/users', async (req, res) => {
try {
const user = await db.createUser(req.body);
res.status(201).json(user);
} catch (err) {
res.status(400).json({ error: err.message });
}
});
app.put('/api/users/:id', async (req, res) => {
try {
const user = await db.updateUser(req.params.id, req.body);
if (!user) {
return res.status(404).json({ error: 'User not found' });
}
res.json(user);
} catch (err) {
res.status(400).json({ error: err.message });
}
});
app.delete('/api/users/:id', async (req, res) => {
try {
await db.deleteUser(req.params.id);
res.status(204).end();
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Error handling middleware
app.use((err, req, res, next) => {
console.error(err.stack);
res.status(500).json({ error: 'Internal server error' });
});
// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});Database Integration
// MongoDB with Mongoose
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/myapp', {
useNewUrlParser: true,
useUnifiedTopology: true
});
const userSchema = new mongoose.Schema({
name: { type: String, required: true },
email: { type: String, required: true, unique: true },
age: Number,
createdAt: { type: Date, default: Date.now }
});
const User = mongoose.model('User', userSchema);
// CRUD operations
async function createUser(data) {
const user = new User(data);
await user.save();
return user;
}
async function getUsers() {
return await User.find();
}
async function getUserById(id) {
return await User.findById(id);
}
async function updateUser(id, data) {
return await User.findByIdAndUpdate(id, data, { new: true });
}
async function deleteUser(id) {
return await User.findByIdAndDelete(id);
}
// PostgreSQL with pg
const { Pool } = require('pg');
const pool = new Pool({
user: 'dbuser',
host: 'localhost',
database: 'mydb',
password: 'password',
port: 5432
});
async function queryUsers() {
const result = await pool.query('SELECT * FROM users');
return result.rows;
}
async function createUser(name, email) {
const result = await pool.query(
'INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *',
[name, email]
);
return result.rows[0];
}Authentication & Authorization
const jwt = require('jsonwebtoken');
const bcrypt = require('bcrypt');
// Hash password
async function hashPassword(password) {
const salt = await bcrypt.genSalt(10);
return await bcrypt.hash(password, salt);
}
// Verify password
async function verifyPassword(password, hash) {
return await bcrypt.compare(password, hash);
}
// Generate JWT
function generateToken(user) {
return jwt.sign(
{ id: user.id, email: user.email },
process.env.JWT_SECRET,
{ expiresIn: '7d' }
);
}
// Verify JWT
function verifyToken(token) {
try {
return jwt.verify(token, process.env.JWT_SECRET);
} catch (err) {
return null;
}
}
// Auth middleware
function authMiddleware(req, res, next) {
const token = req.headers.authorization?.split(' ')[1];
if (!token) {
return res.status(401).json({ error: 'No token provided' });
}
const decoded = verifyToken(token);
if (!decoded) {
return res.status(401).json({ error: 'Invalid token' });
}
req.user = decoded;
next();
}
// Login route
app.post('/api/login', async (req, res) => {
const { email, password } = req.body;
const user = await User.findOne({ email });
if (!user) {
return res.status(401).json({ error: 'Invalid credentials' });
}
const isValid = await verifyPassword(password, user.password);
if (!isValid) {
return res.status(401).json({ error: 'Invalid credentials' });
}
const token = generateToken(user);
res.json({ token, user: { id: user.id, email: user.email } });
});
// Protected route
app.get('/api/profile', authMiddleware, (req, res) => {
res.json({ user: req.user });
});Best Practices
Error Handling
// Async error handling wrapper
function asyncHandler(fn) {
return (req, res, next) => {
Promise.resolve(fn(req, res, next)).catch(next);
};
}
// Usage
app.get('/api/users', asyncHandler(async (req, res) => {
const users = await db.getUsers();
res.json(users);
}));
// Custom error classes
class AppError extends Error {
constructor(message, statusCode) {
super(message);
this.statusCode = statusCode;
this.isOperational = true;
Error.captureStackTrace(this, this.constructor);
}
}
class NotFoundError extends AppError {
constructor(message = 'Resource not found') {
super(message, 404);
}
}
class ValidationError extends AppError {
constructor(message = 'Validation failed') {
super(message, 400);
}
}
// Global error handler
app.use((err, req, res, next) => {
const statusCode = err.statusCode || 500;
const message = err.isOperational ? err.message : 'Internal server error';
// Log error
console.error(err);
// Send response
res.status(statusCode).json({
error: message,
...(process.env.NODE_ENV === 'development' && { stack: err.stack })
});
});
// Unhandled errors
process.on('uncaughtException', (err) => {
console.error('UNCAUGHT EXCEPTION!', err);
process.exit(1);
});
process.on('unhandledRejection', (err) => {
console.error('UNHANDLED REJECTION!', err);
process.exit(1);
});Security
// Use helmet for security headers
const helmet = require('helmet');
app.use(helmet());
// Rate limiting
const rateLimit = require('express-rate-limit');
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100 // limit each IP to 100 requests per windowMs
});
app.use('/api/', limiter);
// CORS
const cors = require('cors');
app.use(cors({
origin: 'https://example.com',
credentials: true
}));
// Input validation
const { body, validationResult } = require('express-validator');
app.post('/api/users',
body('email').isEmail().normalizeEmail(),
body('password').isLength({ min: 8 }),
body('name').trim().notEmpty(),
(req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Process request
}
);
// Sanitize user input
function sanitize(input) {
if (typeof input === 'string') {
return input.replace(/[<>]/g, '');
}
return input;
}
// Environment variables
// Never commit .env files
// Use dotenv for local development
require('dotenv').config();
// Secrets management
const secrets = {
jwtSecret: process.env.JWT_SECRET,
dbPassword: process.env.DB_PASSWORD,
apiKey: process.env.API_KEY
};
// Validate required env vars
const requiredEnvVars = ['JWT_SECRET', 'DB_PASSWORD'];
for (const envVar of requiredEnvVars) {
if (!process.env[envVar]) {
throw new Error(`Missing required environment variable: ${envVar}`);
}
}Performance
// Use compression
const compression = require('compression');
app.use(compression());
// Caching
const NodeCache = require('node-cache');
const cache = new NodeCache({ stdTTL: 600 }); // 10 minutes
app.get('/api/data', async (req, res) => {
const cachedData = cache.get('data');
if (cachedData) {
return res.json(cachedData);
}
const data = await fetchData();
cache.set('data', data);
res.json(data);
});
// Database connection pooling
const pool = new Pool({
max: 20, // Maximum number of connections
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000
});
// Use indexes in databases
// MongoDB
userSchema.index({ email: 1 });
// PostgreSQL
// CREATE INDEX idx_users_email ON users(email);
// Pagination
app.get('/api/users', async (req, res) => {
const page = parseInt(req.query.page) || 1;
const limit = parseInt(req.query.limit) || 10;
const skip = (page - 1) * limit;
const users = await User.find()
.skip(skip)
.limit(limit);
const total = await User.countDocuments();
res.json({
data: users,
page,
limit,
total,
pages: Math.ceil(total / limit)
});
});
// Optimize queries
// Bad - N+1 query problem
const users = await User.find();
for (const user of users) {
user.posts = await Post.find({ userId: user.id });
}
// Good - use joins/population
const users = await User.find().populate('posts');
// Stream large responses
app.get('/api/export', (req, res) => {
res.setHeader('Content-Type', 'application/json');
res.write('[');
let first = true;
User.find().cursor().on('data', (user) => {
if (!first) res.write(',');
res.write(JSON.stringify(user));
first = false;
}).on('end', () => {
res.write(']');
res.end();
});
});Testing
// Using Jest
const request = require('supertest');
const app = require('./app');
describe('GET /api/users', () => {
it('should return all users', async () => {
const res = await request(app)
.get('/api/users')
.expect(200);
expect(Array.isArray(res.body)).toBe(true);
});
});
describe('POST /api/users', () => {
it('should create a new user', async () => {
const userData = {
name: 'John Doe',
email: 'john@example.com'
};
const res = await request(app)
.post('/api/users')
.send(userData)
.expect(201);
expect(res.body.name).toBe(userData.name);
expect(res.body.email).toBe(userData.email);
});
it('should return 400 for invalid data', async () => {
const res = await request(app)
.post('/api/users')
.send({})
.expect(400);
});
});
// Mocking
jest.mock('./database');
const db = require('./database');
test('getUser returns user data', async () => {
db.getUserById.mockResolvedValue({
id: 1,
name: 'John Doe'
});
const user = await getUser(1);
expect(user.name).toBe('John Doe');
});
// Integration tests with test database
beforeAll(async () => {
await mongoose.connect(process.env.TEST_DB_URL);
});
afterAll(async () => {
await mongoose.connection.close();
});
beforeEach(async () => {
await User.deleteMany({});
});Logging
// Using winston
const winston = require('winston');
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
transports: [
new winston.transports.File({ filename: 'error.log', level: 'error' }),
new winston.transports.File({ filename: 'combined.log' })
]
});
if (process.env.NODE_ENV !== 'production') {
logger.add(new winston.transports.Console({
format: winston.format.simple()
}));
}
logger.info('Server started');
logger.error('Error occurred', { error: err.message });
// Request logging
const morgan = require('morgan');
app.use(morgan('combined', {
stream: {
write: (message) => logger.info(message.trim())
}
}));Deployment
// Graceful shutdown
const server = app.listen(PORT);
process.on('SIGTERM', () => {
console.log('SIGTERM received, closing server gracefully');
server.close(() => {
console.log('Server closed');
// Close database connections
mongoose.connection.close(false, () => {
console.log('MongoDB connection closed');
process.exit(0);
});
});
// Force shutdown after 30 seconds
setTimeout(() => {
console.error('Forced shutdown');
process.exit(1);
}, 30000);
});
// Health check endpoint
app.get('/health', (req, res) => {
res.json({ status: 'ok', timestamp: Date.now() });
});
// Process monitoring
const v8 = require('v8');
const os = require('os');
app.get('/metrics', (req, res) => {
const heapStats = v8.getHeapStatistics();
res.json({
memory: {
used: process.memoryUsage().heapUsed,
total: heapStats.total_heap_size,
limit: heapStats.heap_size_limit
},
cpu: process.cpuUsage(),
uptime: process.uptime(),
system: {
freemem: os.freemem(),
totalmem: os.totalmem(),
loadavg: os.loadavg()
}
});
});Summary
This Node.js development skill covers:
1. Core Concepts: Event loop, modules (CommonJS/ES), async patterns, streams, file system, HTTP/HTTPS 2. API Reference: Buffer, Events, Crypto, and core modules 3. Workflow Patterns: REST APIs, database integration, authentication 4. Best Practices: Error handling, security, performance, testing, logging, deployment 5. Real-world Examples: Complete patterns for building production-ready applications
The patterns and examples represent modern Node.js development practices for building scalable, secure, and maintainable backend applications.
Node.js Development Examples
Comprehensive code examples demonstrating Node.js patterns and use cases.
Table of Contents
1. HTTP Server Examples 2. Express.js REST API 3. File Operations 4. Stream Processing 5. Async Patterns 6. CLI Tools 7. Environment Variables 8. Error Handling 9. Database Integration 10. Authentication & Authorization 11. Middleware Patterns 12. Testing Examples 13. WebSocket Real-time 14. Child Processes 15. Cluster Mode 16. Email Sending 17. File Upload 18. Caching Strategies 19. Rate Limiting 20. Deployment & Production
HTTP Server Examples
Basic HTTP Server
const http = require('http');
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('Hello World\n');
});
const PORT = 3000;
server.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}/`);
});HTTP Server with Routing
const http = require('http');
const url = require('url');
const server = http.createServer((req, res) => {
const parsedUrl = url.parse(req.url, true);
const path = parsedUrl.pathname;
const method = req.method;
if (path === '/' && method === 'GET') {
res.writeHead(200, { 'Content-Type': 'text/html' });
res.end('<h1>Welcome Home</h1>');
} else if (path === '/api/data' && method === 'GET') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ message: 'API Data', timestamp: Date.now() }));
} else if (path === '/api/data' && method === 'POST') {
let body = '';
req.on('data', chunk => {
body += chunk.toString();
});
req.on('end', () => {
try {
const data = JSON.parse(body);
res.writeHead(201, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({
success: true,
received: data,
timestamp: Date.now()
}));
} catch (err) {
res.writeHead(400, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'Invalid JSON' }));
}
});
} else {
res.writeHead(404, { 'Content-Type': 'text/plain' });
res.end('404 Not Found');
}
});
server.listen(3000, () => {
console.log('Server running on port 3000');
});HTTPS Server with SSL/TLS
const https = require('https');
const fs = require('fs');
const options = {
key: fs.readFileSync('server-key.pem'),
cert: fs.readFileSync('server-cert.pem')
};
const server = https.createServer(options, (req, res) => {
res.writeHead(200);
res.end('Secure Hello World\n');
});
server.listen(443, () => {
console.log('HTTPS server running on port 443');
});Express.js REST API
Complete REST API Example
const express = require('express');
const app = express();
// Middleware
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
// In-memory database
let users = [
{ id: 1, name: 'Alice', email: 'alice@example.com' },
{ id: 2, name: 'Bob', email: 'bob@example.com' }
];
let nextId = 3;
// Logging middleware
app.use((req, res, next) => {
console.log(`${new Date().toISOString()} - ${req.method} ${req.url}`);
next();
});
// GET all users
app.get('/api/users', (req, res) => {
res.json(users);
});
// GET single user
app.get('/api/users/:id', (req, res) => {
const user = users.find(u => u.id === parseInt(req.params.id));
if (!user) {
return res.status(404).json({ error: 'User not found' });
}
res.json(user);
});
// POST create user
app.post('/api/users', (req, res) => {
const { name, email } = req.body;
if (!name || !email) {
return res.status(400).json({ error: 'Name and email required' });
}
const user = {
id: nextId++,
name,
email
};
users.push(user);
res.status(201).json(user);
});
// PUT update user
app.put('/api/users/:id', (req, res) => {
const user = users.find(u => u.id === parseInt(req.params.id));
if (!user) {
return res.status(404).json({ error: 'User not found' });
}
const { name, email } = req.body;
if (name) user.name = name;
if (email) user.email = email;
res.json(user);
});
// DELETE user
app.delete('/api/users/:id', (req, res) => {
const index = users.findIndex(u => u.id === parseInt(req.params.id));
if (index === -1) {
return res.status(404).json({ error: 'User not found' });
}
users.splice(index, 1);
res.status(204).end();
});
// Error handling middleware
app.use((err, req, res, next) => {
console.error(err.stack);
res.status(500).json({ error: 'Internal server error' });
});
// 404 handler
app.use((req, res) => {
res.status(404).json({ error: 'Route not found' });
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});
module.exports = app;Express Router Organization
// routes/users.js
const express = require('express');
const router = express.Router();
router.get('/', (req, res) => {
res.json({ users: [] });
});
router.get('/:id', (req, res) => {
res.json({ id: req.params.id });
});
router.post('/', (req, res) => {
res.status(201).json(req.body);
});
module.exports = router;
// app.js
const express = require('express');
const usersRouter = require('./routes/users');
const app = express();
app.use('/api/users', usersRouter);
app.listen(3000);File Operations
Read File Examples
const fs = require('fs').promises;
const path = require('path');
// Read text file
async function readTextFile(filename) {
try {
const data = await fs.readFile(filename, 'utf8');
console.log('File contents:', data);
return data;
} catch (err) {
console.error('Error reading file:', err);
throw err;
}
}
// Read JSON file
async function readJSONFile(filename) {
try {
const data = await fs.readFile(filename, 'utf8');
return JSON.parse(data);
} catch (err) {
if (err.code === 'ENOENT') {
console.error('File not found');
} else if (err instanceof SyntaxError) {
console.error('Invalid JSON');
}
throw err;
}
}
// Read multiple files concurrently
async function readMultipleFiles(filenames) {
try {
const promises = filenames.map(f => fs.readFile(f, 'utf8'));
const results = await Promise.all(promises);
return results;
} catch (err) {
console.error('Error reading files:', err);
throw err;
}
}
// Usage
readTextFile('example.txt');
readJSONFile('config.json').then(config => console.log(config));
readMultipleFiles(['file1.txt', 'file2.txt', 'file3.txt']);Write File Examples
const fs = require('fs').promises;
// Write text file
async function writeTextFile(filename, content) {
try {
await fs.writeFile(filename, content, 'utf8');
console.log('File written successfully');
} catch (err) {
console.error('Error writing file:', err);
throw err;
}
}
// Write JSON file (formatted)
async function writeJSONFile(filename, data) {
try {
const json = JSON.stringify(data, null, 2);
await fs.writeFile(filename, json, 'utf8');
console.log('JSON file written successfully');
} catch (err) {
console.error('Error writing JSON file:', err);
throw err;
}
}
// Append to file
async function appendToFile(filename, content) {
try {
await fs.appendFile(filename, content + '\n', 'utf8');
console.log('Content appended successfully');
} catch (err) {
console.error('Error appending to file:', err);
throw err;
}
}
// Atomic write (write to temp file, then rename)
async function atomicWrite(filename, content) {
const tempFile = `${filename}.tmp`;
try {
await fs.writeFile(tempFile, content, 'utf8');
await fs.rename(tempFile, filename);
console.log('File written atomically');
} catch (err) {
// Clean up temp file if it exists
try {
await fs.unlink(tempFile);
} catch {}
throw err;
}
}
// Usage
writeTextFile('output.txt', 'Hello World');
writeJSONFile('data.json', { name: 'John', age: 30 });
appendToFile('log.txt', 'New log entry');
atomicWrite('config.json', JSON.stringify({ version: '1.0' }));Directory Operations
const fs = require('fs').promises;
const path = require('path');
// List directory contents
async function listDirectory(dirPath) {
try {
const files = await fs.readdir(dirPath);
console.log('Files:', files);
return files;
} catch (err) {
console.error('Error reading directory:', err);
throw err;
}
}
// List with file types
async function listDirectoryDetailed(dirPath) {
try {
const entries = await fs.readdir(dirPath, { withFileTypes: true });
const files = [];
const directories = [];
for (const entry of entries) {
if (entry.isFile()) {
files.push(entry.name);
} else if (entry.isDirectory()) {
directories.push(entry.name);
}
}
return { files, directories };
} catch (err) {
console.error('Error:', err);
throw err;
}
}
// Create directory recursively
async function createDirectory(dirPath) {
try {
await fs.mkdir(dirPath, { recursive: true });
console.log('Directory created');
} catch (err) {
console.error('Error creating directory:', err);
throw err;
}
}
// Remove directory recursively
async function removeDirectory(dirPath) {
try {
await fs.rm(dirPath, { recursive: true, force: true });
console.log('Directory removed');
} catch (err) {
console.error('Error removing directory:', err);
throw err;
}
}
// Walk directory tree
async function walkDirectory(dirPath, callback) {
const entries = await fs.readdir(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
if (entry.isFile()) {
await callback(fullPath, 'file');
} else if (entry.isDirectory()) {
await callback(fullPath, 'directory');
await walkDirectory(fullPath, callback);
}
}
}
// Usage
walkDirectory('./src', async (filepath, type) => {
console.log(`${type}: ${filepath}`);
});Stream Processing
Reading Large Files with Streams
const fs = require('fs');
const readline = require('readline');
// Process large file line by line
async function processLargeFile(filename) {
const fileStream = fs.createReadStream(filename);
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
let lineNumber = 0;
for await (const line of rl) {
lineNumber++;
console.log(`Line ${lineNumber}: ${line}`);
}
console.log(`Processed ${lineNumber} lines`);
}
// Stream copy
function copyFile(source, destination) {
return new Promise((resolve, reject) => {
const readStream = fs.createReadStream(source);
const writeStream = fs.createWriteStream(destination);
readStream.pipe(writeStream);
writeStream.on('finish', resolve);
writeStream.on('error', reject);
readStream.on('error', reject);
});
}
// Usage
processLargeFile('large-log.txt');
copyFile('source.txt', 'destination.txt');Transform Streams
const { Transform } = require('stream');
const fs = require('fs');
// Custom transform stream to uppercase text
class UpperCaseTransform extends Transform {
_transform(chunk, encoding, callback) {
this.push(chunk.toString().toUpperCase());
callback();
}
}
// Custom transform to filter lines
class LineFilterTransform extends Transform {
constructor(pattern) {
super();
this.pattern = pattern;
this.buffer = '';
}
_transform(chunk, encoding, callback) {
this.buffer += chunk.toString();
const lines = this.buffer.split('\n');
this.buffer = lines.pop(); // Keep incomplete line in buffer
for (const line of lines) {
if (line.includes(this.pattern)) {
this.push(line + '\n');
}
}
callback();
}
_flush(callback) {
if (this.buffer && this.buffer.includes(this.pattern)) {
this.push(this.buffer + '\n');
}
callback();
}
}
// Usage: Convert file to uppercase
fs.createReadStream('input.txt')
.pipe(new UpperCaseTransform())
.pipe(fs.createWriteStream('output.txt'));
// Usage: Filter log lines containing 'ERROR'
fs.createReadStream('app.log')
.pipe(new LineFilterTransform('ERROR'))
.pipe(fs.createWriteStream('errors.log'));CSV Processing with Streams
const fs = require('fs');
const { Transform } = require('stream');
const readline = require('readline');
class CSVParser extends Transform {
constructor() {
super({ objectMode: true });
this.headers = null;
}
_transform(line, encoding, callback) {
const values = line.split(',').map(v => v.trim());
if (!this.headers) {
this.headers = values;
} else {
const obj = {};
this.headers.forEach((header, index) => {
obj[header] = values[index];
});
this.push(obj);
}
callback();
}
}
async function processCSV(filename) {
const fileStream = fs.createReadStream(filename);
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
const parser = new CSVParser();
parser.on('data', (obj) => {
console.log('Row:', obj);
});
for await (const line of rl) {
parser.write(line);
}
parser.end();
}
processCSV('data.csv');Async Patterns
Promise.all vs Promise.allSettled
// Promise.all - fails fast on first error
async function fetchAllData() {
try {
const [users, posts, comments] = await Promise.all([
fetchUsers(),
fetchPosts(),
fetchComments()
]);
return { users, posts, comments };
} catch (err) {
console.error('One of the requests failed:', err);
throw err;
}
}
// Promise.allSettled - waits for all, reports all results
async function fetchAllDataRobust() {
const results = await Promise.allSettled([
fetchUsers(),
fetchPosts(),
fetchComments()
]);
const data = {};
const errors = {};
results.forEach((result, index) => {
const names = ['users', 'posts', 'comments'];
const name = names[index];
if (result.status === 'fulfilled') {
data[name] = result.value;
} else {
errors[name] = result.reason;
console.error(`Failed to fetch ${name}:`, result.reason);
}
});
return { data, errors };
}Retry Pattern
async function retry(fn, maxAttempts = 3, delay = 1000) {
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
return await fn();
} catch (err) {
if (attempt === maxAttempts) {
throw err;
}
console.log(`Attempt ${attempt} failed, retrying in ${delay}ms...`);
await new Promise(resolve => setTimeout(resolve, delay));
// Exponential backoff
delay *= 2;
}
}
}
// Usage
async function fetchWithRetry() {
return retry(
() => fetch('https://api.example.com/data'),
3,
1000
);
}Rate Limiting / Throttling
class RateLimiter {
constructor(maxRequests, windowMs) {
this.maxRequests = maxRequests;
this.windowMs = windowMs;
this.requests = [];
}
async waitIfNeeded() {
const now = Date.now();
// Remove old requests outside the window
this.requests = this.requests.filter(time => now - time < this.windowMs);
if (this.requests.length >= this.maxRequests) {
const oldestRequest = this.requests[0];
const waitTime = this.windowMs - (now - oldestRequest);
if (waitTime > 0) {
console.log(`Rate limit reached, waiting ${waitTime}ms...`);
await new Promise(resolve => setTimeout(resolve, waitTime));
}
}
this.requests.push(Date.now());
}
async execute(fn) {
await this.waitIfNeeded();
return fn();
}
}
// Usage: 5 requests per second
const limiter = new RateLimiter(5, 1000);
async function makeRequests() {
const urls = Array(20).fill('https://api.example.com/data');
for (const url of urls) {
await limiter.execute(() => fetch(url));
}
}Parallel Processing with Concurrency Limit
async function parallelLimit(tasks, limit) {
const results = [];
const executing = [];
for (const [index, task] of tasks.entries()) {
const promise = Promise.resolve().then(() => task()).then(
result => {
results[index] = { status: 'fulfilled', value: result };
},
error => {
results[index] = { status: 'rejected', reason: error };
}
);
results.push(null);
executing.push(promise);
if (executing.length >= limit) {
await Promise.race(executing);
executing.splice(executing.findIndex(p => p === promise), 1);
}
}
await Promise.all(executing);
return results;
}
// Usage: Process 100 items with max 5 concurrent operations
const tasks = Array(100).fill(0).map((_, i) =>
() => processItem(i)
);
parallelLimit(tasks, 5);CLI Tools
Basic CLI Tool
#!/usr/bin/env node
const args = process.argv.slice(2);
if (args.length === 0) {
console.log('Usage: mycli <command> [options]');
console.log('Commands:');
console.log(' hello [name] - Say hello');
console.log(' version - Show version');
process.exit(0);
}
const command = args[0];
switch (command) {
case 'hello':
const name = args[1] || 'World';
console.log(`Hello, ${name}!`);
break;
case 'version':
const pkg = require('./package.json');
console.log(`Version: ${pkg.version}`);
break;
default:
console.error(`Unknown command: ${command}`);
process.exit(1);
}CLI with Commander.js
#!/usr/bin/env node
const { program } = require('commander');
const pkg = require('./package.json');
program
.name('mycli')
.description('My awesome CLI tool')
.version(pkg.version);
program
.command('init')
.description('Initialize a new project')
.option('-t, --template <type>', 'template type', 'default')
.action((options) => {
console.log(`Initializing project with template: ${options.template}`);
});
program
.command('build')
.description('Build the project')
.option('-w, --watch', 'watch for changes')
.option('-o, --output <dir>', 'output directory', 'dist')
.action((options) => {
console.log(`Building project...`);
console.log(`Output directory: ${options.output}`);
if (options.watch) {
console.log('Watching for changes...');
}
});
program
.command('deploy')
.description('Deploy the project')
.argument('<environment>', 'deployment environment')
.option('--dry-run', 'perform a dry run')
.action((environment, options) => {
if (options.dryRun) {
console.log(`[DRY RUN] Would deploy to ${environment}`);
} else {
console.log(`Deploying to ${environment}...`);
}
});
program.parse();Interactive CLI with Inquirer
const inquirer = require('inquirer');
const fs = require('fs').promises;
async function createProject() {
const answers = await inquirer.prompt([
{
type: 'input',
name: 'name',
message: 'Project name:',
default: 'my-project'
},
{
type: 'list',
name: 'template',
message: 'Choose a template:',
choices: ['basic', 'express', 'react', 'vue']
},
{
type: 'checkbox',
name: 'features',
message: 'Select features:',
choices: [
{ name: 'TypeScript', value: 'typescript' },
{ name: 'ESLint', value: 'eslint' },
{ name: 'Prettier', value: 'prettier' },
{ name: 'Testing (Jest)', value: 'jest' }
]
},
{
type: 'confirm',
name: 'install',
message: 'Install dependencies now?',
default: true
}
]);
console.log('\nCreating project with configuration:');
console.log(JSON.stringify(answers, null, 2));
// Create project directory
await fs.mkdir(answers.name);
console.log(`Created directory: ${answers.name}`);
// Create package.json
const packageJson = {
name: answers.name,
version: '1.0.0',
template: answers.template,
features: answers.features
};
await fs.writeFile(
`${answers.name}/package.json`,
JSON.stringify(packageJson, null, 2)
);
console.log('Project created successfully!');
}
createProject();Environment Variables
Using dotenv
// .env file
// PORT=3000
// NODE_ENV=development
// DATABASE_URL=mongodb://localhost/myapp
// JWT_SECRET=your-secret-key
// API_KEY=abc123
// Load environment variables
require('dotenv').config();
const config = {
port: process.env.PORT || 3000,
env: process.env.NODE_ENV || 'development',
database: {
url: process.env.DATABASE_URL
},
jwt: {
secret: process.env.JWT_SECRET
},
api: {
key: process.env.API_KEY
}
};
// Validate required environment variables
const requiredEnvVars = ['DATABASE_URL', 'JWT_SECRET'];
for (const envVar of requiredEnvVars) {
if (!process.env[envVar]) {
console.error(`Missing required environment variable: ${envVar}`);
process.exit(1);
}
}
module.exports = config;Environment-specific Configuration
// config/index.js
const development = {
port: 3000,
database: {
host: 'localhost',
port: 27017,
name: 'myapp_dev'
},
logging: {
level: 'debug'
}
};
const production = {
port: process.env.PORT,
database: {
url: process.env.DATABASE_URL
},
logging: {
level: 'error'
}
};
const test = {
port: 3001,
database: {
host: 'localhost',
port: 27017,
name: 'myapp_test'
},
logging: {
level: 'silent'
}
};
const configs = {
development,
production,
test
};
const env = process.env.NODE_ENV || 'development';
module.exports = configs[env];Error Handling
Custom Error Classes
// errors/AppError.js
class AppError extends Error {
constructor(message, statusCode) {
super(message);
this.statusCode = statusCode;
this.isOperational = true;
Error.captureStackTrace(this, this.constructor);
}
}
class NotFoundError extends AppError {
constructor(resource = 'Resource') {
super(`${resource} not found`, 404);
}
}
class ValidationError extends AppError {
constructor(message = 'Validation failed') {
super(message, 400);
}
}
class UnauthorizedError extends AppError {
constructor(message = 'Unauthorized') {
super(message, 401);
}
}
class ForbiddenError extends AppError {
constructor(message = 'Forbidden') {
super(message, 403);
}
}
module.exports = {
AppError,
NotFoundError,
ValidationError,
UnauthorizedError,
ForbiddenError
};Error Handler Middleware
const { AppError } = require('./errors/AppError');
// Async wrapper to catch errors in async route handlers
const asyncHandler = (fn) => {
return (req, res, next) => {
Promise.resolve(fn(req, res, next)).catch(next);
};
};
// Error logging
function logError(err) {
console.error({
message: err.message,
stack: err.stack,
statusCode: err.statusCode,
timestamp: new Date().toISOString()
});
}
// Error handler middleware
function errorHandler(err, req, res, next) {
logError(err);
// Operational errors (expected)
if (err.isOperational) {
return res.status(err.statusCode).json({
error: err.message
});
}
// Programming errors or unknown errors
res.status(500).json({
error: process.env.NODE_ENV === 'production'
? 'Internal server error'
: err.message,
...(process.env.NODE_ENV === 'development' && { stack: err.stack })
});
}
// Usage in Express app
app.get('/api/users/:id', asyncHandler(async (req, res) => {
const user = await User.findById(req.params.id);
if (!user) {
throw new NotFoundError('User');
}
res.json(user);
}));
app.use(errorHandler);Database Integration
MongoDB with Mongoose
const mongoose = require('mongoose');
// Connect to MongoDB
async function connectDB() {
try {
await mongoose.connect(process.env.MONGODB_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
console.log('MongoDB connected');
} catch (err) {
console.error('MongoDB connection error:', err);
process.exit(1);
}
}
// Define schema
const userSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'Name is required'],
trim: true,
minlength: 2,
maxlength: 50
},
email: {
type: String,
required: [true, 'Email is required'],
unique: true,
lowercase: true,
match: [/^\S+@\S+\.\S+$/, 'Please provide a valid email']
},
age: {
type: Number,
min: 0,
max: 120
},
role: {
type: String,
enum: ['user', 'admin'],
default: 'user'
},
createdAt: {
type: Date,
default: Date.now
}
});
// Add indexes
userSchema.index({ email: 1 });
// Add virtual property
userSchema.virtual('isAdult').get(function() {
return this.age >= 18;
});
// Add instance method
userSchema.methods.getPublicProfile = function() {
return {
id: this._id,
name: this.name,
email: this.email
};
};
// Add static method
userSchema.statics.findByEmail = function(email) {
return this.findOne({ email });
};
const User = mongoose.model('User', userSchema);
// CRUD operations
async function createUser(userData) {
const user = new User(userData);
await user.save();
return user;
}
async function getAllUsers() {
return await User.find().select('-__v');
}
async function getUserById(id) {
return await User.findById(id);
}
async function updateUser(id, updates) {
return await User.findByIdAndUpdate(
id,
updates,
{ new: true, runValidators: true }
);
}
async function deleteUser(id) {
return await User.findByIdAndDelete(id);
}
// Complex queries
async function searchUsers(query) {
const { name, minAge, maxAge, role, sort = 'createdAt' } = query;
const filter = {};
if (name) {
filter.name = { $regex: name, $options: 'i' };
}
if (minAge || maxAge) {
filter.age = {};
if (minAge) filter.age.$gte = minAge;
if (maxAge) filter.age.$lte = maxAge;
}
if (role) {
filter.role = role;
}
return await User.find(filter).sort(sort);
}
module.exports = {
connectDB,
User,
createUser,
getAllUsers,
getUserById,
updateUser,
deleteUser,
searchUsers
};PostgreSQL with pg
const { Pool } = require('pg');
// Create connection pool
const pool = new Pool({
user: process.env.DB_USER,
host: process.env.DB_HOST,
database: process.env.DB_NAME,
password: process.env.DB_PASSWORD,
port: process.env.DB_PORT || 5432,
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000
});
// Test connection
pool.on('connect', () => {
console.log('Connected to PostgreSQL');
});
pool.on('error', (err) => {
console.error('Unexpected error on idle client', err);
process.exit(-1);
});
// CRUD operations
async function createUser(name, email, age) {
const query = {
text: 'INSERT INTO users(name, email, age) VALUES($1, $2, $3) RETURNING *',
values: [name, email, age]
};
const result = await pool.query(query);
return result.rows[0];
}
async function getAllUsers() {
const result = await pool.query('SELECT * FROM users ORDER BY created_at DESC');
return result.rows;
}
async function getUserById(id) {
const query = {
text: 'SELECT * FROM users WHERE id = $1',
values: [id]
};
const result = await pool.query(query);
return result.rows[0];
}
async function updateUser(id, updates) {
const { name, email, age } = updates;
const query = {
text: 'UPDATE users SET name = $1, email = $2, age = $3 WHERE id = $4 RETURNING *',
values: [name, email, age, id]
};
const result = await pool.query(query);
return result.rows[0];
}
async function deleteUser(id) {
const query = {
text: 'DELETE FROM users WHERE id = $1',
values: [id]
};
await pool.query(query);
}
// Transaction example
async function transferMoney(fromAccountId, toAccountId, amount) {
const client = await pool.connect();
try {
await client.query('BEGIN');
// Deduct from source account
await client.query(
'UPDATE accounts SET balance = balance - $1 WHERE id = $2',
[amount, fromAccountId]
);
// Add to destination account
await client.query(
'UPDATE accounts SET balance = balance + $1 WHERE id = $2',
[amount, toAccountId]
);
await client.query('COMMIT');
console.log('Transfer successful');
} catch (err) {
await client.query('ROLLBACK');
console.error('Transfer failed:', err);
throw err;
} finally {
client.release();
}
}
module.exports = {
pool,
createUser,
getAllUsers,
getUserById,
updateUser,
deleteUser,
transferMoney
};Authentication & Authorization
JWT Authentication
const jwt = require('jsonwebtoken');
const bcrypt = require('bcrypt');
const JWT_SECRET = process.env.JWT_SECRET;
const JWT_EXPIRES_IN = '7d';
// Hash password
async function hashPassword(password) {
const saltRounds = 10;
return await bcrypt.hash(password, saltRounds);
}
// Verify password
async function verifyPassword(password, hash) {
return await bcrypt.compare(password, hash);
}
// Generate access token
function generateAccessToken(user) {
return jwt.sign(
{
id: user.id,
email: user.email,
role: user.role
},
JWT_SECRET,
{ expiresIn: JWT_EXPIRES_IN }
);
}
// Generate refresh token
function generateRefreshToken(user) {
return jwt.sign(
{ id: user.id },
JWT_SECRET,
{ expiresIn: '30d' }
);
}
// Verify token
function verifyToken(token) {
try {
return jwt.verify(token, JWT_SECRET);
} catch (err) {
return null;
}
}
// Authentication middleware
function authMiddleware(req, res, next) {
const authHeader = req.headers.authorization;
if (!authHeader || !authHeader.startsWith('Bearer ')) {
return res.status(401).json({ error: 'No token provided' });
}
const token = authHeader.split(' ')[1];
const decoded = verifyToken(token);
if (!decoded) {
return res.status(401).json({ error: 'Invalid or expired token' });
}
req.user = decoded;
next();
}
// Role-based authorization middleware
function requireRole(...roles) {
return (req, res, next) => {
if (!req.user) {
return res.status(401).json({ error: 'Unauthorized' });
}
if (!roles.includes(req.user.role)) {
return res.status(403).json({ error: 'Forbidden' });
}
next();
};
}
// Register route
app.post('/api/auth/register', async (req, res) => {
try {
const { name, email, password } = req.body;
// Check if user exists
const existing = await User.findOne({ email });
if (existing) {
return res.status(400).json({ error: 'Email already registered' });
}
// Hash password
const hashedPassword = await hashPassword(password);
// Create user
const user = await User.create({
name,
email,
password: hashedPassword
});
// Generate tokens
const accessToken = generateAccessToken(user);
const refreshToken = generateRefreshToken(user);
res.status(201).json({
user: {
id: user.id,
name: user.name,
email: user.email
},
accessToken,
refreshToken
});
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Login route
app.post('/api/auth/login', async (req, res) => {
try {
const { email, password } = req.body;
// Find user
const user = await User.findOne({ email });
if (!user) {
return res.status(401).json({ error: 'Invalid credentials' });
}
// Verify password
const isValid = await verifyPassword(password, user.password);
if (!isValid) {
return res.status(401).json({ error: 'Invalid credentials' });
}
// Generate tokens
const accessToken = generateAccessToken(user);
const refreshToken = generateRefreshToken(user);
res.json({
user: {
id: user.id,
name: user.name,
email: user.email,
role: user.role
},
accessToken,
refreshToken
});
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Protected route
app.get('/api/profile', authMiddleware, (req, res) => {
res.json({ user: req.user });
});
// Admin-only route
app.get('/api/admin/users', authMiddleware, requireRole('admin'), async (req, res) => {
const users = await User.find();
res.json(users);
});
module.exports = {
hashPassword,
verifyPassword,
generateAccessToken,
generateRefreshToken,
verifyToken,
authMiddleware,
requireRole
};Middleware Patterns
Logging Middleware
// Request logging
function requestLogger(req, res, next) {
const start = Date.now();
res.on('finish', () => {
const duration = Date.now() - start;
console.log({
method: req.method,
url: req.url,
status: res.statusCode,
duration: `${duration}ms`,
timestamp: new Date().toISOString()
});
});
next();
}
app.use(requestLogger);Validation Middleware
const { body, validationResult } = require('express-validator');
// Validation rules
const userValidationRules = [
body('name')
.trim()
.notEmpty().withMessage('Name is required')
.isLength({ min: 2, max: 50 }).withMessage('Name must be 2-50 characters'),
body('email')
.trim()
.notEmpty().withMessage('Email is required')
.isEmail().withMessage('Must be a valid email')
.normalizeEmail(),
body('password')
.notEmpty().withMessage('Password is required')
.isLength({ min: 8 }).withMessage('Password must be at least 8 characters')
.matches(/\d/).withMessage('Password must contain a number')
.matches(/[a-z]/).withMessage('Password must contain a lowercase letter')
.matches(/[A-Z]/).withMessage('Password must contain an uppercase letter'),
body('age')
.optional()
.isInt({ min: 0, max: 120 }).withMessage('Age must be between 0 and 120')
];
// Validation middleware
function validate(req, res, next) {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
next();
}
// Usage
app.post('/api/users', userValidationRules, validate, async (req, res) => {
const user = await createUser(req.body);
res.status(201).json(user);
});CORS Middleware
const cors = require('cors');
// Basic CORS
app.use(cors());
// Custom CORS
const corsOptions = {
origin: function (origin, callback) {
const allowedOrigins = [
'https://example.com',
'https://app.example.com'
];
if (!origin || allowedOrigins.includes(origin)) {
callback(null, true);
} else {
callback(new Error('Not allowed by CORS'));
}
},
credentials: true,
methods: ['GET', 'POST', 'PUT', 'DELETE'],
allowedHeaders: ['Content-Type', 'Authorization']
};
app.use(cors(corsOptions));Testing Examples
Unit Tests with Jest
// sum.js
function sum(a, b) {
return a + b;
}
module.exports = { sum };
// sum.test.js
const { sum } = require('./sum');
describe('sum function', () => {
test('adds 1 + 2 to equal 3', () => {
expect(sum(1, 2)).toBe(3);
});
test('adds positive numbers', () => {
expect(sum(10, 20)).toBe(30);
});
test('adds negative numbers', () => {
expect(sum(-5, -10)).toBe(-15);
});
test('adds mixed numbers', () => {
expect(sum(-5, 10)).toBe(5);
});
});API Integration Tests
const request = require('supertest');
const app = require('./app');
const { connectDB, User } = require('./db');
beforeAll(async () => {
await connectDB();
});
afterAll(async () => {
await mongoose.connection.close();
});
beforeEach(async () => {
await User.deleteMany({});
});
describe('User API', () => {
describe('POST /api/users', () => {
it('should create a new user', async () => {
const userData = {
name: 'John Doe',
email: 'john@example.com',
password: 'Password123'
};
const response = await request(app)
.post('/api/users')
.send(userData)
.expect(201)
.expect('Content-Type', /json/);
expect(response.body).toHaveProperty('id');
expect(response.body.name).toBe(userData.name);
expect(response.body.email).toBe(userData.email);
expect(response.body).not.toHaveProperty('password');
});
it('should return 400 for invalid data', async () => {
const response = await request(app)
.post('/api/users')
.send({ name: 'John' })
.expect(400);
expect(response.body).toHaveProperty('errors');
});
});
describe('GET /api/users/:id', () => {
it('should return user by id', async () => {
const user = await User.create({
name: 'John Doe',
email: 'john@example.com',
password: 'hashed'
});
const response = await request(app)
.get(`/api/users/${user.id}`)
.expect(200);
expect(response.body.name).toBe(user.name);
});
it('should return 404 for non-existent user', async () => {
const response = await request(app)
.get('/api/users/507f1f77bcf86cd799439011')
.expect(404);
});
});
});Mocking in Tests
// userService.js
const axios = require('axios');
async function getUserFromAPI(id) {
const response = await axios.get(`https://api.example.com/users/${id}`);
return response.data;
}
module.exports = { getUserFromAPI };
// userService.test.js
jest.mock('axios');
const axios = require('axios');
const { getUserFromAPI } = require('./userService');
describe('getUserFromAPI', () => {
it('should fetch user data', async () => {
const mockUser = { id: 1, name: 'John' };
axios.get.mockResolvedValue({ data: mockUser });
const user = await getUserFromAPI(1);
expect(user).toEqual(mockUser);
expect(axios.get).toHaveBeenCalledWith('https://api.example.com/users/1');
});
it('should handle errors', async () => {
axios.get.mockRejectedValue(new Error('Network error'));
await expect(getUserFromAPI(1)).rejects.toThrow('Network error');
});
});WebSocket Real-time
WebSocket Server
const WebSocket = require('ws');
const http = require('http');
const express = require('express');
const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });
// Store connected clients
const clients = new Set();
wss.on('connection', (ws, req) => {
console.log('New client connected');
clients.add(ws);
// Send welcome message
ws.send(JSON.stringify({
type: 'welcome',
message: 'Connected to WebSocket server',
clientCount: clients.size
}));
// Broadcast client count to all
broadcast({
type: 'clientCount',
count: clients.size
});
// Handle messages
ws.on('message', (data) => {
try {
const message = JSON.parse(data);
console.log('Received:', message);
// Echo message to all clients
broadcast({
type: 'message',
data: message,
timestamp: Date.now()
});
} catch (err) {
console.error('Invalid message:', err);
}
});
// Handle disconnect
ws.on('close', () => {
console.log('Client disconnected');
clients.delete(ws);
broadcast({
type: 'clientCount',
count: clients.size
});
});
// Handle errors
ws.on('error', (err) => {
console.error('WebSocket error:', err);
});
});
// Broadcast to all connected clients
function broadcast(data) {
const message = JSON.stringify(data);
clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.send(message);
}
});
}
// Heartbeat to keep connections alive
setInterval(() => {
clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.ping();
}
});
}, 30000);
server.listen(3000, () => {
console.log('Server running on port 3000');
});Chat Application
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
const rooms = new Map();
class ChatRoom {
constructor(name) {
this.name = name;
this.clients = new Map();
}
addClient(ws, username) {
this.clients.set(ws, username);
this.broadcast({
type: 'userJoined',
username,
userCount: this.clients.size
});
}
removeClient(ws) {
const username = this.clients.get(ws);
this.clients.delete(ws);
if (username) {
this.broadcast({
type: 'userLeft',
username,
userCount: this.clients.size
});
}
}
broadcast(data, exclude = null) {
const message = JSON.stringify(data);
this.clients.forEach((username, client) => {
if (client !== exclude && client.readyState === WebSocket.OPEN) {
client.send(message);
}
});
}
sendMessage(ws, username, text) {
this.broadcast({
type: 'message',
username,
text,
timestamp: Date.now()
});
}
}
wss.on('connection', (ws) => {
let currentRoom = null;
let username = null;
ws.on('message', (data) => {
try {
const message = JSON.parse(data);
switch (message.type) {
case 'join':
username = message.username;
currentRoom = message.room;
if (!rooms.has(currentRoom)) {
rooms.set(currentRoom, new ChatRoom(currentRoom));
}
const room = rooms.get(currentRoom);
room.addClient(ws, username);
ws.send(JSON.stringify({
type: 'joined',
room: currentRoom,
users: Array.from(room.clients.values())
}));
break;
case 'message':
if (currentRoom && username) {
rooms.get(currentRoom).sendMessage(ws, username, message.text);
}
break;
}
} catch (err) {
console.error('Error:', err);
}
});
ws.on('close', () => {
if (currentRoom && rooms.has(currentRoom)) {
const room = rooms.get(currentRoom);
room.removeClient(ws);
if (room.clients.size === 0) {
rooms.delete(currentRoom);
}
}
});
});Child Processes
Executing Commands
const { exec, execFile, spawn } = require('child_process');
const util = require('util');
const execPromise = util.promisify(exec);
// exec - run shell command
async function runShellCommand(command) {
try {
const { stdout, stderr } = await execPromise(command);
console.log('Output:', stdout);
if (stderr) console.error('Errors:', stderr);
return stdout;
} catch (err) {
console.error('Command failed:', err);
throw err;
}
}
// execFile - run executable directly (safer)
async function runExecutable(file, args) {
return new Promise((resolve, reject) => {
execFile(file, args, (err, stdout, stderr) => {
if (err) {
reject(err);
return;
}
resolve({ stdout, stderr });
});
});
}
// spawn - for long-running processes or large output
function runLongProcess(command, args) {
const child = spawn(command, args);
child.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
child.stderr.on('data', (data) => {
console.error(`stderr: ${data}`);
});
child.on('close', (code) => {
console.log(`Process exited with code ${code}`);
});
child.on('error', (err) => {
console.error('Failed to start process:', err);
});
return child;
}
// Usage examples
runShellCommand('ls -la');
runExecutable('node', ['--version']);
runLongProcess('npm', ['install']);Worker Processes
// parent.js
const { fork } = require('child_process');
const child = fork('./worker.js');
child.on('message', (msg) => {
console.log('Message from child:', msg);
});
child.send({ task: 'compute', data: [1, 2, 3, 4, 5] });
child.on('exit', (code) => {
console.log(`Child exited with code ${code}`);
});
// worker.js
process.on('message', (msg) => {
console.log('Message from parent:', msg);
if (msg.task === 'compute') {
const result = msg.data.reduce((sum, num) => sum + num, 0);
process.send({ result });
process.exit(0);
}
});Cluster Mode
Using Node.js Cluster
const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
// Listen for dying workers
cluster.on('exit', (worker, code, signal) => {
console.log(`Worker ${worker.process.pid} died`);
console.log('Starting a new worker');
cluster.fork();
});
// Listen for online workers
cluster.on('online', (worker) => {
console.log(`Worker ${worker.process.pid} is online`);
});
} else {
// Workers can share any TCP connection
const server = http.createServer((req, res) => {
res.writeHead(200);
res.end(`Process ${process.pid} handled request\n`);
});
server.listen(8000);
console.log(`Worker ${process.pid} started`);
}Email Sending
Using Nodemailer
const nodemailer = require('nodemailer');
// Create transporter
const transporter = nodemailer.createTransporter({
host: process.env.SMTP_HOST,
port: process.env.SMTP_PORT,
secure: true,
auth: {
user: process.env.SMTP_USER,
pass: process.env.SMTP_PASSWORD
}
});
// Send email
async function sendEmail(to, subject, html) {
try {
const info = await transporter.sendMail({
from: '"My App" <noreply@myapp.com>',
to,
subject,
html
});
console.log('Message sent:', info.messageId);
return info;
} catch (err) {
console.error('Error sending email:', err);
throw err;
}
}
// Send welcome email
async function sendWelcomeEmail(user) {
const html = `
<h1>Welcome ${user.name}!</h1>
<p>Thanks for signing up.</p>
`;
await sendEmail(user.email, 'Welcome to My App', html);
}
// Send password reset email
async function sendPasswordResetEmail(user, resetToken) {
const resetUrl = `https://myapp.com/reset-password?token=${resetToken}`;
const html = `
<h1>Password Reset</h1>
<p>Click the link below to reset your password:</p>
<a href="${resetUrl}">Reset Password</a>
<p>This link expires in 1 hour.</p>
`;
await sendEmail(user.email, 'Password Reset Request', html);
}
module.exports = {
sendEmail,
sendWelcomeEmail,
sendPasswordResetEmail
};File Upload
Multer File Upload
const express = require('express');
const multer = require('multer');
const path = require('path');
const fs = require('fs').promises;
const app = express();
// Configure storage
const storage = multer.diskStorage({
destination: async (req, file, cb) => {
const uploadDir = './uploads';
await fs.mkdir(uploadDir, { recursive: true });
cb(null, uploadDir);
},
filename: (req, file, cb) => {
const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9);
cb(null, file.fieldname + '-' + uniqueSuffix + path.extname(file.originalname));
}
});
// File filter
const fileFilter = (req, file, cb) => {
const allowedTypes = /jpeg|jpg|png|gif|pdf/;
const extname = allowedTypes.test(path.extname(file.originalname).toLowerCase());
const mimetype = allowedTypes.test(file.mimetype);
if (extname && mimetype) {
cb(null, true);
} else {
cb(new Error('Only images and PDFs are allowed'));
}
};
const upload = multer({
storage,
limits: {
fileSize: 5 * 1024 * 1024 // 5MB
},
fileFilter
});
// Single file upload
app.post('/api/upload', upload.single('file'), (req, res) => {
if (!req.file) {
return res.status(400).json({ error: 'No file uploaded' });
}
res.json({
message: 'File uploaded successfully',
filename: req.file.filename,
size: req.file.size,
path: req.file.path
});
});
// Multiple files upload
app.post('/api/upload-multiple', upload.array('files', 5), (req, res) => {
if (!req.files || req.files.length === 0) {
return res.status(400).json({ error: 'No files uploaded' });
}
const files = req.files.map(file => ({
filename: file.filename,
size: file.size,
path: file.path
}));
res.json({
message: 'Files uploaded successfully',
files
});
});
// Error handling
app.use((err, req, res, next) => {
if (err instanceof multer.MulterError) {
if (err.code === 'LIMIT_FILE_SIZE') {
return res.status(400).json({ error: 'File too large' });
}
if (err.code === 'LIMIT_FILE_COUNT') {
return res.status(400).json({ error: 'Too many files' });
}
}
res.status(500).json({ error: err.message });
});
app.listen(3000);Caching Strategies
In-Memory Cache with node-cache
const NodeCache = require('node-cache');
const cache = new NodeCache({
stdTTL: 600, // 10 minutes default
checkperiod: 120 // Check for expired keys every 2 minutes
});
// Cache wrapper for async functions
function cacheWrapper(key, ttl) {
return function(target, propertyName, descriptor) {
const originalMethod = descriptor.value;
descriptor.value = async function(...args) {
const cacheKey = `${key}:${JSON.stringify(args)}`;
// Try to get from cache
const cached = cache.get(cacheKey);
if (cached !== undefined) {
console.log('Cache hit:', cacheKey);
return cached;
}
// Execute original method
console.log('Cache miss:', cacheKey);
const result = await originalMethod.apply(this, args);
// Store in cache
cache.set(cacheKey, result, ttl);
return result;
};
return descriptor;
};
}
// Manual caching
async function getUserData(userId) {
const cacheKey = `user:${userId}`;
// Check cache
const cached = cache.get(cacheKey);
if (cached) {
return cached;
}
// Fetch from database
const user = await User.findById(userId);
// Store in cache for 5 minutes
cache.set(cacheKey, user, 300);
return user;
}
// Invalidate cache
function invalidateUserCache(userId) {
cache.del(`user:${userId}`);
}
// Clear all cache
function clearCache() {
cache.flushAll();
}
module.exports = {
cache,
getUserData,
invalidateUserCache,
clearCache
};Rate Limiting
Express Rate Limit
const rateLimit = require('express-rate-limit');
// Basic rate limit
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // Limit each IP to 100 requests per windowMs
message: 'Too many requests, please try again later',
standardHeaders: true,
legacyHeaders: false
});
// Apply to all requests
app.use('/api/', limiter);
// Strict rate limit for authentication
const authLimiter = rateLimit({
windowMs: 15 * 60 * 1000,
max: 5,
skipSuccessfulRequests: true,
message: 'Too many login attempts, please try again later'
});
app.post('/api/auth/login', authLimiter, loginHandler);
// Custom key generator (by user ID instead of IP)
const userLimiter = rateLimit({
windowMs: 60 * 1000,
max: 10,
keyGenerator: (req) => {
return req.user?.id || req.ip;
}
});
app.use('/api/user/', authMiddleware, userLimiter);Deployment & Production
PM2 Ecosystem File
// ecosystem.config.js
module.exports = {
apps: [{
name: 'my-app',
script: './index.js',
instances: 'max',
exec_mode: 'cluster',
env: {
NODE_ENV: 'development',
PORT: 3000
},
env_production: {
NODE_ENV: 'production',
PORT: 8080
},
error_file: './logs/err.log',
out_file: './logs/out.log',
log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
merge_logs: true,
autorestart: true,
watch: false,
max_memory_restart: '1G',
max_restarts: 10,
min_uptime: '10s'
}]
};Graceful Shutdown
const express = require('express');
const app = express();
const server = app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});
// Graceful shutdown
process.on('SIGTERM', gracefulShutdown);
process.on('SIGINT', gracefulShutdown);
function gracefulShutdown() {
console.log('Received shutdown signal, closing server gracefully...');
server.close(async () => {
console.log('HTTP server closed');
try {
// Close database connections
await mongoose.connection.close();
console.log('Database connection closed');
// Close other connections (Redis, etc.)
await redisClient.quit();
console.log('Redis connection closed');
console.log('Graceful shutdown complete');
process.exit(0);
} catch (err) {
console.error('Error during shutdown:', err);
process.exit(1);
}
});
// Force shutdown after 30 seconds
setTimeout(() => {
console.error('Forced shutdown after timeout');
process.exit(1);
}, 30000);
}Health Check Endpoint
app.get('/health', async (req, res) => {
const health = {
uptime: process.uptime(),
timestamp: Date.now(),
status: 'ok'
};
try {
// Check database connection
await mongoose.connection.db.admin().ping();
health.database = 'connected';
} catch (err) {
health.database = 'disconnected';
health.status = 'error';
}
const statusCode = health.status === 'ok' ? 200 : 503;
res.status(statusCode).json(health);
});This comprehensive examples file covers all major Node.js development patterns and use cases, providing production-ready code that developers can adapt for their projects.
Node.js Development Skill
A comprehensive skill for building modern Node.js applications covering backend APIs, CLI tools, microservices, and real-time applications.
Overview
Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine that enables server-side JavaScript execution. It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient for building scalable network applications.
This skill provides comprehensive guidance on:
- Event-driven architecture and the event loop
- Asynchronous programming patterns (callbacks, promises, async/await)
- Stream processing for efficient data handling
- File system operations and path management
- HTTP/HTTPS server creation and request handling
- Process management and environment configuration
- Security best practices and error handling
- Performance optimization and testing strategies
Why Node.js?
Node.js excels at:
1. I/O-Intensive Applications: Non-blocking I/O makes it perfect for applications with many concurrent connections 2. Real-time Applications: WebSockets and Server-Sent Events enable real-time bidirectional communication 3. API Development: Fast, lightweight, and perfect for RESTful APIs and GraphQL servers 4. Microservices: Small footprint and quick startup time ideal for microservice architecture 5. Developer Productivity: JavaScript everywhere (frontend and backend) reduces context switching 6. Rich Ecosystem: npm provides access to over 1 million packages 7. Streaming Data: Built-in stream support for handling large files and data processing
Getting Started
Installation
Using Official Installer
1. Visit nodejs.org 2. Download the LTS (Long Term Support) version 3. Run the installer for your platform
Using Node Version Manager (Recommended)
macOS/Linux (nvm):
# Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
# Install Node.js LTS
nvm install --lts
# Use specific version
nvm use 18
# Set default version
nvm alias default 18Windows (nvm-windows):
# Download installer from: https://github.com/coreybutler/nvm-windows/releases
# Then install Node.js
nvm install lts
nvm use ltsVerify Installation
node --version
npm --versionYour First Node.js Program
Create a file hello.js:
console.log('Hello, Node.js!');Run it:
node hello.jsCreating a Project
# Create project directory
mkdir my-nodejs-app
cd my-nodejs-app
# Initialize package.json
npm init -y
# Install dependencies
npm install express
# Create main file
touch index.jsBasic HTTP Server
// index.js
const http = require('http');
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('Hello World\n');
});
const PORT = 3000;
server.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}/`);
});Run the server:
node index.jsVisit http://localhost:3000 in your browser.
Express.js Hello World
const express = require('express');
const app = express();
const PORT = 3000;
app.get('/', (req, res) => {
res.send('Hello World!');
});
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});Core Concepts
Event Loop
The event loop is what allows Node.js to perform non-blocking I/O operations despite JavaScript being single-threaded. It's the mechanism that handles asynchronous callbacks.
Key Points:
- JavaScript is single-threaded but uses asynchronous callbacks
- The event loop continuously checks for tasks to execute
- I/O operations are offloaded to the system kernel when possible
- Callbacks are executed when operations complete
Non-Blocking I/O
Node.js uses non-blocking I/O calls, allowing it to support thousands of concurrent connections without the overhead of thread management.
// Blocking (synchronous)
const data = fs.readFileSync('file.txt'); // Waits for file read
console.log(data);
// Non-blocking (asynchronous)
fs.readFile('file.txt', (err, data) => {
if (err) throw err;
console.log(data);
});
console.log('This executes immediately');Module System
Node.js uses modules to organize code into reusable components. It supports both CommonJS (traditional) and ES Modules (modern).
CommonJS:
// Export
module.exports = { add, subtract };
// Import
const math = require('./math');ES Modules:
// Export
export { add, subtract };
// Import
import { add, subtract } from './math.js';NPM (Node Package Manager)
NPM is the world's largest software registry. It allows you to install, share, and manage dependencies.
# Install package
npm install express
# Install as dev dependency
npm install --save-dev jest
# Install globally
npm install -g nodemon
# Uninstall package
npm uninstall express
# Update packages
npm update
# List installed packages
npm list
# Check for outdated packages
npm outdatedCommon Use Cases
1. REST API Development
Build RESTful APIs with Express.js:
const express = require('express');
const app = express();
app.use(express.json());
app.get('/api/users', (req, res) => {
res.json({ users: [] });
});
app.post('/api/users', (req, res) => {
const user = req.body;
res.status(201).json(user);
});
app.listen(3000);2. Real-time Applications
Use WebSockets for real-time communication:
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', (ws) => {
ws.on('message', (message) => {
// Broadcast to all clients
wss.clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.send(message);
}
});
});
});3. CLI Tools
Create command-line tools:
#!/usr/bin/env node
const args = process.argv.slice(2);
const command = args[0];
switch (command) {
case 'help':
console.log('Available commands: help, version');
break;
case 'version':
console.log('v1.0.0');
break;
default:
console.log('Unknown command');
}4. File Processing
Process files efficiently with streams:
const fs = require('fs');
const readStream = fs.createReadStream('large-file.txt');
const writeStream = fs.createWriteStream('output.txt');
readStream.pipe(writeStream);
writeStream.on('finish', () => {
console.log('File processing complete');
});Project Structure
A typical Node.js project structure:
my-app/
├── node_modules/ # Dependencies
├── src/ # Source code
│ ├── controllers/ # Route handlers
│ ├── models/ # Data models
│ ├── routes/ # API routes
│ ├── middleware/ # Custom middleware
│ ├── utils/ # Utility functions
│ └── config/ # Configuration files
├── tests/ # Test files
├── public/ # Static files
├── .env # Environment variables (not committed)
├── .gitignore # Git ignore file
├── package.json # Project metadata and dependencies
├── package-lock.json # Locked dependency versions
└── index.js # Entry pointEnvironment Configuration
Use environment variables for configuration:
.env file:
PORT=3000
NODE_ENV=development
DATABASE_URL=mongodb://localhost/myapp
JWT_SECRET=your-secret-keyLoad with dotenv:
require('dotenv').config();
const port = process.env.PORT || 3000;
const dbUrl = process.env.DATABASE_URL;Development Tools
Essential Tools
1. nodemon - Auto-restart on file changes
npm install -g nodemon
nodemon index.js2. ESLint - Code linting
npm install --save-dev eslint
npx eslint --init3. Prettier - Code formatting
npm install --save-dev prettier
npx prettier --write .4. Jest - Testing framework
npm install --save-dev jest
npm testpackage.json Scripts
{
"scripts": {
"start": "node index.js",
"dev": "nodemon index.js",
"test": "jest",
"lint": "eslint .",
"format": "prettier --write ."
}
}Run scripts:
npm start
npm run dev
npm testBest Practices
1. Error Handling
Always handle errors properly:
// Async/await
try {
const data = await fetchData();
} catch (err) {
console.error('Error:', err);
}
// Promises
fetchData()
.then(data => process(data))
.catch(err => console.error('Error:', err));
// Process-level error handling
process.on('uncaughtException', (err) => {
console.error('Uncaught exception:', err);
process.exit(1);
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled rejection:', reason);
process.exit(1);
});2. Security
- Never commit secrets or API keys
- Use environment variables for sensitive data
- Validate and sanitize user input
- Use HTTPS in production
- Keep dependencies updated
- Use security headers (helmet.js)
- Implement rate limiting
- Use parameterized queries to prevent SQL injection
3. Performance
- Use asynchronous methods instead of synchronous
- Implement caching strategies
- Use connection pooling for databases
- Enable compression
- Optimize database queries
- Use clustering for multi-core systems
- Monitor memory usage and performance
4. Code Organization
- Follow the single responsibility principle
- Use modules to organize code
- Keep functions small and focused
- Use meaningful variable and function names
- Add comments for complex logic
- Follow a consistent coding style
Debugging
Using Node.js Inspector
# Start with inspector
node --inspect index.js
# Debug from the start
node --inspect-brk index.jsOpen chrome://inspect in Chrome to debug.
Using VS Code
Create .vscode/launch.json:
{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/index.js"
}
]
}Press F5 to start debugging.
Console Debugging
console.log('Variable:', variable);
console.error('Error:', error);
console.table(arrayOfObjects);
console.time('operation');
// ... code to measure
console.timeEnd('operation');Testing
Unit Testing with Jest
// math.js
function add(a, b) {
return a + b;
}
module.exports = { add };
// math.test.js
const { add } = require('./math');
test('adds 1 + 2 to equal 3', () => {
expect(add(1, 2)).toBe(3);
});Integration Testing
const request = require('supertest');
const app = require('./app');
describe('GET /api/users', () => {
it('responds with json', async () => {
const response = await request(app)
.get('/api/users')
.expect('Content-Type', /json/)
.expect(200);
});
});Deployment
Popular Platforms
1. Heroku
heroku create
git push heroku main2. Vercel
npm install -g vercel
vercel3. AWS (Elastic Beanstalk, Lambda) 4. Google Cloud Platform 5. DigitalOcean 6. Railway 7. Render
Production Checklist
- [ ] Set NODE_ENV=production
- [ ] Use process manager (PM2)
- [ ] Enable logging
- [ ] Set up monitoring
- [ ] Configure error tracking (Sentry)
- [ ] Use HTTPS
- [ ] Set up CI/CD
- [ ] Configure auto-scaling
- [ ] Set up database backups
- [ ] Implement health checks
Using PM2
# Install PM2
npm install -g pm2
# Start application
pm2 start index.js
# Start with name
pm2 start index.js --name "my-app"
# Start in cluster mode
pm2 start index.js -i max
# Monitor
pm2 monit
# List processes
pm2 list
# Restart
pm2 restart my-app
# Stop
pm2 stop my-app
# View logs
pm2 logs
# Save process list
pm2 save
# Auto-start on boot
pm2 startupResources
Official Documentation
Popular Frameworks
- Express.js - Fast, unopinionated web framework
- Fastify - High-performance web framework
- NestJS - Progressive TypeScript framework
- Koa - Lightweight web framework
Learning Resources
Community
Next Steps
1. Explore the SKILL.md file for comprehensive API reference and patterns 2. Review EXAMPLES.md for detailed code examples 3. Build a real project (REST API, CLI tool, or real-time app) 4. Learn TypeScript for better type safety 5. Explore advanced topics (worker threads, cluster mode, streams) 6. Contribute to open-source Node.js projects
License
This skill is provided as-is for educational and development purposes.
Node.js Development Skill
A comprehensive skill for building modern Node.js applications covering backend APIs, CLI tools, microservices, and real-time applications.
Overview
Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine that enables server-side JavaScript execution. It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient for building scalable network applications.
This skill provides comprehensive guidance on:
- Event-driven architecture and the event loop
- Asynchronous programming patterns (callbacks, promises, async/await)
- Stream processing for efficient data handling
- File system operations and path management
- HTTP/HTTPS server creation and request handling
- Process management and environment configuration
- Security best practices and error handling
- Performance optimization and testing strategies
Why Node.js?
Node.js excels at:
1. I/O-Intensive Applications: Non-blocking I/O makes it perfect for applications with many concurrent connections 2. Real-time Applications: WebSockets and Server-Sent Events enable real-time bidirectional communication 3. API Development: Fast, lightweight, and perfect for RESTful APIs and GraphQL servers 4. Microservices: Small footprint and quick startup time ideal for microservice architecture 5. Developer Productivity: JavaScript everywhere (frontend and backend) reduces context switching 6. Rich Ecosystem: npm provides access to over 1 million packages 7. Streaming Data: Built-in stream support for handling large files and data processing
Getting Started
Installation
Using Official Installer
1. Visit nodejs.org 2. Download the LTS (Long Term Support) version 3. Run the installer for your platform
Using Node Version Manager (Recommended)
macOS/Linux (nvm):
# Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
# Install Node.js LTS
nvm install --lts
# Use specific version
nvm use 18
# Set default version
nvm alias default 18Windows (nvm-windows):
# Download installer from: https://github.com/coreybutler/nvm-windows/releases
# Then install Node.js
nvm install lts
nvm use ltsVerify Installation
node --version
npm --versionYour First Node.js Program
Create a file hello.js:
console.log('Hello, Node.js!');Run it:
node hello.jsCreating a Project
# Create project directory
mkdir my-nodejs-app
cd my-nodejs-app
# Initialize package.json
npm init -y
# Install dependencies
npm install express
# Create main file
touch index.jsBasic HTTP Server
// index.js
const http = require('http');
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('Hello World\n');
});
const PORT = 3000;
server.listen(PORT, () => {
console.log(`Server running at http://localhost:${PORT}/`);
});Run the server:
node index.jsVisit http://localhost:3000 in your browser.
Express.js Hello World
const express = require('express');
const app = express();
const PORT = 3000;
app.get('/', (req, res) => {
res.send('Hello World!');
});
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});Core Concepts
Event Loop
The event loop is what allows Node.js to perform non-blocking I/O operations despite JavaScript being single-threaded. It's the mechanism that handles asynchronous callbacks.
Key Points:
- JavaScript is single-threaded but uses asynchronous callbacks
- The event loop continuously checks for tasks to execute
- I/O operations are offloaded to the system kernel when possible
- Callbacks are executed when operations complete
Non-Blocking I/O
Node.js uses non-blocking I/O calls, allowing it to support thousands of concurrent connections without the overhead of thread management.
// Blocking (synchronous)
const data = fs.readFileSync('file.txt'); // Waits for file read
console.log(data);
// Non-blocking (asynchronous)
fs.readFile('file.txt', (err, data) => {
if (err) throw err;
console.log(data);
});
console.log('This executes immediately');Module System
Node.js uses modules to organize code into reusable components. It supports both CommonJS (traditional) and ES Modules (modern).
CommonJS:
// Export
module.exports = { add, subtract };
// Import
const math = require('./math');ES Modules:
// Export
export { add, subtract };
// Import
import { add, subtract } from './math.js';NPM (Node Package Manager)
NPM is the world's largest software registry. It allows you to install, share, and manage dependencies.
# Install package
npm install express
# Install as dev dependency
npm install --save-dev jest
# Install globally
npm install -g nodemon
# Uninstall package
npm uninstall express
# Update packages
npm update
# List installed packages
npm list
# Check for outdated packages
npm outdatedCommon Use Cases
1. REST API Development
Build RESTful APIs with Express.js:
const express = require('express');
const app = express();
app.use(express.json());
app.get('/api/users', (req, res) => {
res.json({ users: [] });
});
app.post('/api/users', (req, res) => {
const user = req.body;
res.status(201).json(user);
});
app.listen(3000);2. Real-time Applications
Use WebSockets for real-time communication:
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
wss.on('connection', (ws) => {
ws.on('message', (message) => {
// Broadcast to all clients
wss.clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.send(message);
}
});
});
});3. CLI Tools
Create command-line tools:
#!/usr/bin/env node
const args = process.argv.slice(2);
const command = args[0];
switch (command) {
case 'help':
console.log('Available commands: help, version');
break;
case 'version':
console.log('v1.0.0');
break;
default:
console.log('Unknown command');
}4. File Processing
Process files efficiently with streams:
const fs = require('fs');
const readStream = fs.createReadStream('large-file.txt');
const writeStream = fs.createWriteStream('output.txt');
readStream.pipe(writeStream);
writeStream.on('finish', () => {
console.log('File processing complete');
});Project Structure
A typical Node.js project structure:
my-app/
├── node_modules/ # Dependencies
├── src/ # Source code
│ ├── controllers/ # Route handlers
│ ├── models/ # Data models
│ ├── routes/ # API routes
│ ├── middleware/ # Custom middleware
│ ├── utils/ # Utility functions
│ └── config/ # Configuration files
├── tests/ # Test files
├── public/ # Static files
├── .env # Environment variables (not committed)
├── .gitignore # Git ignore file
├── package.json # Project metadata and dependencies
├── package-lock.json # Locked dependency versions
└── index.js # Entry pointEnvironment Configuration
Use environment variables for configuration:
.env file:
PORT=3000
NODE_ENV=development
DATABASE_URL=mongodb://localhost/myapp
JWT_SECRET=your-secret-keyLoad with dotenv:
require('dotenv').config();
const port = process.env.PORT || 3000;
const dbUrl = process.env.DATABASE_URL;Development Tools
Essential Tools
1. nodemon - Auto-restart on file changes
npm install -g nodemon
nodemon index.js2. ESLint - Code linting
npm install --save-dev eslint
npx eslint --init3. Prettier - Code formatting
npm install --save-dev prettier
npx prettier --write .4. Jest - Testing framework
npm install --save-dev jest
npm testpackage.json Scripts
{
"scripts": {
"start": "node index.js",
"dev": "nodemon index.js",
"test": "jest",
"lint": "eslint .",
"format": "prettier --write ."
}
}Run scripts:
npm start
npm run dev
npm testBest Practices
1. Error Handling
Always handle errors properly:
// Async/await
try {
const data = await fetchData();
} catch (err) {
console.error('Error:', err);
}
// Promises
fetchData()
.then(data => process(data))
.catch(err => console.error('Error:', err));
// Process-level error handling
process.on('uncaughtException', (err) => {
console.error('Uncaught exception:', err);
process.exit(1);
});
process.on('unhandledRejection', (reason, promise) => {
console.error('Unhandled rejection:', reason);
process.exit(1);
});2. Security
- Never commit secrets or API keys
- Use environment variables for sensitive data
- Validate and sanitize user input
- Use HTTPS in production
- Keep dependencies updated
- Use security headers (helmet.js)
- Implement rate limiting
- Use parameterized queries to prevent SQL injection
3. Performance
- Use asynchronous methods instead of synchronous
- Implement caching strategies
- Use connection pooling for databases
- Enable compression
- Optimize database queries
- Use clustering for multi-core systems
- Monitor memory usage and performance
4. Code Organization
- Follow the single responsibility principle
- Use modules to organize code
- Keep functions small and focused
- Use meaningful variable and function names
- Add comments for complex logic
- Follow a consistent coding style
Debugging
Using Node.js Inspector
# Start with inspector
node --inspect index.js
# Debug from the start
node --inspect-brk index.jsOpen chrome://inspect in Chrome to debug.
Using VS Code
Create .vscode/launch.json:
{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/index.js"
}
]
}Press F5 to start debugging.
Console Debugging
console.log('Variable:', variable);
console.error('Error:', error);
console.table(arrayOfObjects);
console.time('operation');
// ... code to measure
console.timeEnd('operation');Testing
Unit Testing with Jest
// math.js
function add(a, b) {
return a + b;
}
module.exports = { add };
// math.test.js
const { add } = require('./math');
test('adds 1 + 2 to equal 3', () => {
expect(add(1, 2)).toBe(3);
});Integration Testing
const request = require('supertest');
const app = require('./app');
describe('GET /api/users', () => {
it('responds with json', async () => {
const response = await request(app)
.get('/api/users')
.expect('Content-Type', /json/)
.expect(200);
});
});Deployment
Popular Platforms
1. Heroku
heroku create
git push heroku main2. Vercel
npm install -g vercel
vercel3. AWS (Elastic Beanstalk, Lambda) 4. Google Cloud Platform 5. DigitalOcean 6. Railway 7. Render
Production Checklist
- [ ] Set NODE_ENV=production
- [ ] Use process manager (PM2)
- [ ] Enable logging
- [ ] Set up monitoring
- [ ] Configure error tracking (Sentry)
- [ ] Use HTTPS
- [ ] Set up CI/CD
- [ ] Configure auto-scaling
- [ ] Set up database backups
- [ ] Implement health checks
Using PM2
# Install PM2
npm install -g pm2
# Start application
pm2 start index.js
# Start with name
pm2 start index.js --name "my-app"
# Start in cluster mode
pm2 start index.js -i max
# Monitor
pm2 monit
# List processes
pm2 list
# Restart
pm2 restart my-app
# Stop
pm2 stop my-app
# View logs
pm2 logs
# Save process list
pm2 save
# Auto-start on boot
pm2 startupResources
Official Documentation
Popular Frameworks
- Express.js - Fast, unopinionated web framework
- Fastify - High-performance web framework
- NestJS - Progressive TypeScript framework
- Koa - Lightweight web framework
Learning Resources
Community
Next Steps
1. Explore the SKILL.md file for comprehensive API reference and patterns 2. Review EXAMPLES.md for detailed code examples 3. Build a real project (REST API, CLI tool, or real-time app) 4. Learn TypeScript for better type safety 5. Explore advanced topics (worker threads, cluster mode, streams) 6. Contribute to open-source Node.js projects
License
This skill is provided as-is for educational and development purposes.