In this tutorial, I’ll show you how to build a video encoder with Node.js. We will use express.js on the backend to run a web server, multer for uploading files and socket.io for broadcasting real-time encoding progress to the user. We will use handbrake-js to encode videos which under the hood, spawns HandBrakeCLI process and lets us listen to events. We will use React to build our Frontend where a user can upload videos, view real-time encoding progress, download and view their converted videos.
Getting Started
I’m going to split this article into three parts. In the first part, we will create a simple web server and setup basic user identification. In the second part, we will write our uploader components and implement backend logic for uploading videos using multer. In the last part, we will build an encoder component and implement backend logic for encoding videos and broadcast real-time progress to the user through WebSockets.
Here’s a demo of how our app will look and function at the end of this tutorial.
Web Server
Let’s start by creating a bare-bones HTTP server. In your working directory create a server.js file and add this code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
const express = require('express'); const app = express(); const server = require('http').Server(app); const io = require('socket.io')(server); const config = require('config'); const PORT = config.get('port'); app.set('views', __dirname + '/views'); app.set('view engine', 'ejs'); app.use('/', express.static(__dirname + '/public')); app.get('*', (req, res) => { res.render('app'); }); server.listen(PORT, () => console.log('Server running on Port: '+ PORT)); |
I’m passing HTTP server to socket.io to hook them together and run on the same port. Since we are using react and react-router on the frontend, we will bootstrap our SPA inside app view and render it against every incoming request. Add an app.ejs file under your views directory and add this code.
1 2 3 4 5 6 7 8 9 10 11 |
<!doctype html> <html lang="en"> <head> <title>nverter</title> <link rel="stylesheet" href="/style.css"> </head> <body> <div id="root"></div> </body> <script src="/bundle.js"></script> </html> |
User Identification
Since we are not authenticating users, we will simply create a cookie in their browser to identify them. Let’s create a middleware to store a unique id cookie in the user’s browser.
1 2 3 4 5 |
const cookieParser = require('cookie-parser'); const cookieMiddleware = require('./middleware/userCookie'); app.use(cookieParser()); app.use(cookieMiddleware()); |
Create userCookie.js file under the middleware directory and add this code.
1 2 3 4 5 6 7 8 9 10 11 |
const uuidv4 = require('uuid/v4'); module.exports = (options = null) => { return (req, res, next) => { const cookie = req.cookies._uid; if (cookie === undefined) { res.cookie('_uid', uuidv4(), { maxAge : (3600000 * 24) * 30 , httpOnly: false, domain : '127.0.0.1' }); } next() } } |
Express will run this middleware for every incoming request. It will create a unique id cookie (_uid) in the user’s browser if it doesn’t exist.
Uploader
Create a Uploader.js file under the src directory. We will place all our components under this directory along with the CSS files.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
class Uploader extends Component{ constructor(props) { super(props); this.state = { file: null, uploading: false, progress : 0, upload_ext : null, convert_ext : '', allowed_types : [ 'webm', 'mkv', 'flv', 'ogg', 'avi', 'mov' , 'wmv', 'mp4', 'm4v', 'm4p', 'mpeg', '3gp', '3g2' ] }; this.fileInput = React.createRef(); this.selectFile = this.initFileUpload.bind(this); this.uploadFile = this.uploadFile.bind(this); this.cancelUpload = this.cancelUpload.bind(this); this.handleChange = this.setConversionFormat.bind(this); } render(){ return( <div className="uploader"> {!this.state.uploading ? <div> <div> {this.state.file ? <button onClick={this.uploadFile}>Upload File</button> : <button onClick={this.selectFile}>Select Video File</button> } {this.state.file && <button onClick={this.cancelUpload}>Cancel</button> } </div> {this.state.file && <div> <select value={this.state.convert_ext} onChange={this.handleChange}> <option value=""> Convert To </option> { this.state.allowed_types.map((ext) => { if(ext !== this.state.upload_ext){ return <option key={ext} value={ext}>{ext}</option> } }) } </select> </div> } <input type="file" name="file" className="form-control-file" ref={this.fileInput} onChange={this.onFileChange.bind(this)}/> </div> : <Progress title="Uploading, please wait" progress={this.state.progress}/> } </div> ); } } |
In Uploader component, I’ve set up an initial state and bound event handlers. Initially, we will display a select video file button, onClick it will trigger a click on a hidden file input and opens up a file dialog. On file select, we are calling onFileChange method. Let’s create it.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
onFileChange(e) { if(!e.target.files.length){ return; } let file = e.target.files[0], ext = this.getFileExtension(file.name); if (this.validateFile(ext)) { this.setState({ file: file, upload_ext : ext }) }else{ toastr.error('Error: Invalid file format') } } |
In this method, we are validating if a user has selected a file. We will then parse out selected file extension and check if exists in the allowed_types array. Add these two methods to your component.
1 2 3 4 5 6 7 |
getFileExtension(name){ return /(?:\.([^.]+))?$/.exec(name)[1]; } validateFile(ext) { return this.state.allowed_types.includes(ext); } |
getFileExtension method will parse out file extension from its name using regex and validateFile method will lookout for extension in the allowed_types array in component’s state. If the file passes these validation checks, we will update file and upload_ext state on our component. After updating the state, we’ll display Upload and Cancel buttons. A select drop-down will be populated with available formats for video encoding. We will filter out the extension of the file that the user has selected. On cancel, we will reset the component state by calling cancelUpload method.
1 2 3 4 5 6 7 8 9 10 |
cancelUpload(e){ this.setState({ file: null, uploading: false, progress : 0, upload_ext : null, convert_ext : '', }); this.fileInput.current.value = ''; } |
We have an uploadFile method for handling Upload button clicks.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
uploadFile(e){ if(this.state.file && this.state.convert_ext){ this.setState({ uploading: true, }); let data = new FormData(); data.append('file', this.state.file); data.append('convert_ext', this.state.convert_ext); post('/upload', data, { onUploadProgress: (progressEvent) => { let percentCompleted = Math.round( (progressEvent.loaded * 100) / progressEvent.total ); this.setState({ progress : percentCompleted }); } }) .then(res => { let file = res.data; if(file.uploaded){ this.props.initEncoding(file.path, this.state.convert_ext); } }) .catch(err => { console.log(err); }); }else{ toastr.error('Error: Select a conversion format') } } |
In this method, we will validate if there’s a file present and the user has selected an extension from the drop-down input. After passing this validation check, we will create a FormData() object and append file and ext to it. We will also set uploading state to true which will render Progress component. Progress is a functional component that takes in percentage value as a prop and renders a progress bar.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
export default (props) => { return ( <div className="progress-container"> <h2> {props.title} </h2> <progress value={props.progress} max="100" className="progress"/> <div> {props.progress} % </div> </div> ) } |
We are using axios to upload the file. We are passing onUploadProgress method with config to axios which listens to progress event. We will use it to calculate the upload percentage and update component progress state which will result in re-rendering of Progress component.
Let’s configure multer storage and handle upload request on the backend.
1 2 3 4 5 6 7 8 9 10 11 12 |
const multer = require('multer'); const storage = multer.diskStorage({ destination: function (req, file, cb) { cb(null, path.resolve(__dirname , 'uploads')) }, filename: function (req, file, cb) { cb(null, Date.now() + '_' + file.originalname) } }); const upload = multer({storage : storage}); |
We have configured multer to store uploaded file in the uploads directory. A date will be concatenated with uploaded file names.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
app.post('/upload', upload.single('file'), (req, res) => { if(req.file){ let video = req.file, user = req.cookies._uid; let upload_path = video.path, user_upload = path.join(__dirname, '/uploads/', user), move_path = path.join(__dirname, '/uploads/' , user , video.filename); fs.exists(user_upload, (exists) => { if(!exists){ fs.mkdir(user_upload,function(err){ if (err) { return console.error(err); } moveUploadedFileToUserDir( upload_path, move_path, video.filename, res ); }); }else{ moveUploadedFileToUserDir( upload_path, move_path, video.filename, res ); } }); createUserEncodeDir(user); }else{ res.json({ uploaded : false }) } }); let moveUploadedFileToUserDir = (upload_path, move_path, filename, res) =>{ fs.rename(upload_path, move_path, (err) => { if (err) throw err; res.json({ uploaded : true, path : filename }); }); }; let createUserEncodeDir = (user) => { let dir = path.join(__dirname, '/encoded/', user); fs.exists(dir, (exists) => { if(!exists) { fs.mkdir(dir, function (err) { if (err) { return console.error(err); } }); } }); }; |
multer middleware will make upload file available through req.file. After upload, we will move the file to the user’s upload directory. We will retrieve the user’s unique id from req.cookies and use it to create a user directory and move uploaded file to it. On success, we will return filename as a response.
1 2 3 |
if(file.uploaded){ this.props.initEncoding(file.path, this.state.convert_ext); } |
initEncoding method is being passed from App component. Let’s create our App component and define this method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
class App extends Component { constructor(props){ super(props); this.state = { encoder : false, uploader : true, file: '', convert_ext : '' } } initEncoding(file, ext) { this.setState({ encoder: true, uploader: false, file: file, convert_ext : ext }); } clearEncode(e = null){ this.setState({ encoder : false, uploader : true, file: '', convert_ext : '' }) } render() { return ( <div className="App"> <Navbar/> <Route exact path="/" render={(props) => ( <div className="wrapper"> {this.state.uploader ? ( <Uploader initEncoding={this.initEncoding.bind(this)}/> ) : <Encoder file={this.state.file} convert_ext={this.state.convert_ext} newEncode={this.clearEncode.bind(this)}/> } </div> )}/> <Route exact path="/encodes" component={History}/> </div> ); } } |
In our App component, we will initially render Uploader component and when it calls our initEncoding prop method, we will update its state. We are passing file and convert_ext as props to our Encoder component.
Encoder
After Uploader component calls initEncoding prop method on successful upload, App component will render Encoder component.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
export default class Encoder extends Component { constructor(props){ super(props); this.state = { file : props.file, encoded_file : '', convert_ext : props.convert_ext, progress : 0, eta : '', } } componentDidMount(){ this.socket = socketIOClient('ws://127.0.0.1:3000'); this.socket.emit('encode', { file : this.state.file, user : Cookie('_uid'), convert_ext : this.state.convert_ext }); this.socket.on('progress', function (data) { this.setState({ progress : data.percentage, eta : data.eta }); }.bind(this)); this.socket.on('complete', function (data) { this.setState({ encoded_file : data.encoded_file }); toastr.success('Encoding complete'); }.bind(this)); } componentWillUnmount(){ this.socket.disconnect(); this.props.newEncode(); } render(){ let filename = this.state.file; return ( <div className="encoder"> <h3> {filename.substring(filename.indexOf('_') + 1)} <br/> <small> ETA : {this.state.eta.trim().length ? this.state.eta : 'calculating ... ' } </small> </h3> <Progress title="" progress={this.state.progress}/> {this.state.encoded_file ? ( <div> <a href={ '/encoded/' + Cookie('_uid') + '/' + this.state.encoded_file} download> <button>Download</button> </a> <button onClick={this.props.newEncode}>New Upload</button> </div> ) : <button onClick={this.props.newEncode}>Cancel Encoding</button>} </div> ) } } |
After mounting, we will open a WebSocket connection and emit an encode event. Let’s handle this socket connection and event on the backend. Add this code to your server.js file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
io.on('connection', (socket) => { socket.on('encode', (data) => { let handbrake, completed = false, file = data.file, user = data.user, convert_ext = data.convert_ext, input = path.join(__dirname, '/uploads/', user, '/' + file), encoded_file = file + '_to_.' + convert_ext, output = path.join(__dirname, '/encoded/', user , '/', encoded_file); handbrake = hbjs.spawn({ input: input, output : output, preset : 'Universal' }) .on('progress', progress => { socket.emit('progress',{ percentage : progress.percentComplete, eta : progress.eta }); }) .on('complete', () => { completed = true; socket.emit('complete',{ encoded_file : encoded_file }); }); socket.on('disconnect', () => { if(!completed){ console.log('Not completed'); handbrake.cancel(); deleteVideo(input); deleteVideo(output); } }); }); }); let deleteVideo = (path) => { fs.unlink(path, (err) => { if (err) throw err; }); }; |
We will spawn a separate HandBrakeCLI process for every connected socket. If socket disconnects before completion, we will stop handbrake process and delete uploaded and incomplete encoded video. On handbrake progress event, we’ll emit progress event and pass progress percentage and eta with it. On complete event, we will emit a complete event with the encoded file name. On the front end, we are listening to these WebSocket events and update encoded_file state on completion. After updating state, a download button will be displayed to the user for downloading their encoded file.
Since we are using react-router, we will render History component for /encode path and render a Link in Navbar that points to it.
<Route exact path="/encodes" component={History}/>
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
export default class History extends React.Component{ constructor(props){ super(props); this.state = { encodes : [] } } componentDidMount(){ get('/history') .then(res => { console.log(res.data); this.setState({ encodes : res.data }); }); } render(){ let uid = Cookie('_uid'); return ( <div className="encodes"> <h2> Encode History </h2> {this.state.encodes.map(encode => { return ( <div key={encode} className="encode"> {encode.substring(encode.indexOf('_') + 1)} <a href={'/encoded/' + uid + '/' + encode} className="download" download> Download </a> </div> ) })} {!this.state.encodes.length && ( <div className="message"> No Encodes Found </div> )} </div> ) } } |
When it renders, we will make a get request to retrieve user’s encoded files. We will render them in a list with download buttons. Add this router handler on the backend to return encoded files.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
app.get('/history', (req, res) => { let dir = path.join(__dirname, 'encoded', req.cookies._uid); fs.exists(dir, (exists) => { if(exists){ fs.readdir(dir, (err, files) => { if (err) throw err; res.json(files.reverse()); }); }else{ res.json([]); } }); }); |
A lot of small details like CSS styles and webpack config are missing from this tutorial. I’ve created a Github repository with complete code and setup instructions. If you get any errors, please mention them in comments or open a Github issue.
Getting error when ran code from Github link
The “path” argument must be of type string. Received type undefined
Where, upon doing what and how do I regenerate it? Node version?
I received the same error.
So I followed the directions per github and everything was working until the ‘npm run start’ command. Up until this point, the front end code built with no errors, and the system let me know there were multiple mongod processes running. I killed them all because I got an internal server error(500) on the browser and only ran ‘node server.js’ I then received the error mentioned in the comment above ‘The “path” argument must be of type string. Received type undefined’. So I’m thinking that I need to correct the ‘path.join’ syntax? I’m creating the upload folder fine but I am not able to see an uploaded file the history in the ‘encode history’ button. Also, nothing is being converter. Any help or direction would be greatly appreciated. I’m very interested in this project.
Me too.
It is looking for socker user that is comming from Cookies, just make a small fix for this:
js
io.on('connection', (socket) => {
socket.on('encode', (data) => {
let handbrake,
completed = false,
file = data.file,
user = socket.handshake.headers?.cookie.substring(5, 41), // <==== Change it
...
Use http://127.0.0.1:port instead of localhost:port or lvh.me:port. There is hardcoded domain : ‘127.0.0.1’ in userCookie.js file.
Great post … i will try to adapt it in a Nestjs Backend I am building now.
Love your guides man, keep them coming!