How to Upload to S3 Folder Nodejs

1. Habitation
2. Magazine
3. Uploading files to AWS S3 using Nodejs

Uploading files to AWS S3 using Nodejs

Past 06th Mar, 2018

• Category: Web log
• 457

AWS S3. A identify where you can store files. That'southward what most of you already know virtually information technology. S3 is one of the older service provided past Amazon, before the days of revolutionary Lambda functions and game changing Alexa Skills.You can store almost whatsoever blazon of files from doc to pdf, and of size ranging from 0B to 5TB.

Co-ordinate to their official docs, AWS S3 is,

"S3 provides comprehensive security and compliance capabilities that meet even the most stringent regulatory requirements. It gives customers flexibility in the way they manage data for cost optimization, access control, and compliance. " - AWS Docs

If I attempt to put it in simple terms, AWS S3, is an object based storage arrangement where every file your store is saved equally object non file. There are many big tech-wigs, which uses S3, and Dropbox is one of them. Recently, Dropbox starts saving metadata of their file in their ain service only they are notwithstanding saving main data in S3. Why? Well, S3 is non that expensive and it'south 99.ix% bachelor. Plus, you become the change to use services similar Glacier which can save data and charge almost $0.01 per GB.

So far, if I accept gotten your attention, and yous're thinking how to use S3 in my nodejs application. Well, y'all don't take to wait for long.

AWS has official package which exposes S3 apis for node js apps and makes it besides easy for developers to access S3 from their apps.

Source: https://youtu.be/FLolHgKRTKg

In adjacent few steps, I volition guide you to build a nodejs based app, which can write any file to AWS S3.

1. Prepare node app

A basic node app usually have 2 file, package.json (for dependencies)  and a starter file (like app.js, index.js, server.js).

Y'all can use your Os's file manager or your favourite IDE to create projection just I usually prefer CLI. And so, let's go into our beat out and follow these commands:

mkdir s3-contacts-upload-demo
cd s3-contacts-upload-demo
impact index.js .gitignore
npm init

If y'all didn't become any error in above commands, you lot volition have a folder by the proper noun of s3-contacts-upload-demo, three files in it, package.json, .gitignore andindex.json. You can apply this file to add together file in your .gitignore file, so that these won't become committed to github or some other version control.

npm init creates a package.json, which have project's details, you can but hit enter in your beat for default values if you lot wish.

2. Install dependencies

Let'southward first by installing the NPM package:

npm install --salve aws-sdk

After successful installation of this parcel, you tin can likewise cheque your package.json file, it will have aws-sdk listed in "dependencies" field.

This npm package can be used to access any AWS service from your nodejs app, and here we volition apply it for S3.

3. Import packages

One time installed, import the package in your lawmaking:

const fs = require('fs');
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
  accessKeyId: procedure.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});

As y'all can notice, we have also imported fs package which volition be used to write file data in this app. We are as well using environment variables, to fix AWS access and secret admission key, as it is a bad practice to put them on version control similar Github or SVN.

Now, you accept your S3 instance, which can admission all the buckets in your AWS business relationship.

4. Pass bucket data and write business logic

Below is a simple paradigm of how to upload file to S3. Here, Bucket is name of your bucket and key is proper noun of subfolder. And then, if your saucepan name is "exam-bucket" and yous want to save file in "test-bucket/folder/subfolder/file.csv", and then value of Central should be "older/subfolder/file.csv".

Note: S3 is object based storage and non file based. So, fifty-fifty in AWS console you can run across nested folders, behind the scene they never become saved like that. Every object has two fields: Key and Value. Key hither is simply name of file and Value is data which is getting stored.

And then, if a bucket "bucket1" has key "key1/key2/file.mp3", you can visualize it like this:

{
"bucket1": {
"key1/key2/file.mp3": "<mp3-data>"
}
}

Beneath is simple snippe to upload a file,using Fundamental and BucketName.

const params = {
 Bucket: 'saucepan',
 Key: 'fundamental',
 Torso: stream
};
s3.upload(params, role(err, data) {
console.log(err, data);
});

5. File to upload to S3

First, create a file, allow'south say contacts.csv and write some data in it.

Above is some dummy information for you to go started. Now y'all take a contacts.csv file, allow's read it using fs module and salve it to S3.

S3 upload method returns error and information in callback, where data field contains location, bucket and primal of uploaded file. For consummate API reference, refer their official docs.

Now run this app past following command:

AWS_ACCESS_KEY=<your_access_key>
AWS_SECRET_ACCESS_KEY=<your_secret_key> node index.js

We have passed our AWS keys every bit environment variables.

If yous don't become whatever error in above snippet, your bucket should accept the file.

And that's it, under 30 lines of code yous uploaded a file to AWS S3.

You can observe the full project hither.

There are lot many things you can practise with this bundle like

• List out buckets and objects

• Set permissions on saucepan

• Create, get or delete bucket and much more

I would highly recommend you to through this medico for APIs, they are very well explained with parameters you can pass to each API and response format they render.

I promise you constitute this post useful and please do let u.s. know in case of any question or query.

• Node JS

Useful Links

christensenbrobbegreare.blogspot.com

Source: https://www.zeolearn.com/magazine/uploading-files-to-aws-s3-using-nodejs

Share this post