How to upload files on AWS S3 by using curl, without having command line aws or other tool

March 31, 2020

Introduction

There were few files that I need to take backup from a machine that I recently launched. The machine neither had aws command line utility, nor any other code by which I could upload my files on aws s3.

Curl the savior

I already wrote few useful commands for curl

By using curl, you can actually upload the file on aws s3. The requirement is that you must have the access key and the secret key. Lets write a shell script.

# about the file
file_to_upload=<file you want to upload>
bucket=<your s3 bucket name>
filepath="/${bucket}/${file_to_upload}"

# metadata
contentType="application/x-compressed-tar"
dateValue=`date -R`
signature_string="PUT\n\n${contentType}\n${dateValue}\n${filepath}"

#s3 keys
s3_access_key=<your s3 access key>
s3_secret_key=<your s3 secret key>

#prepare signature hash to be sent in Authorization header
signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${s3_secret_key} -binary | base64`

# actual curl command to do PUT operation on s3
curl -X PUT -T "${file_to_upload}" \
  -H "Host: ${bucket}.s3.amazonaws.com" \
  -H "Date: ${dateValue}" \
  -H "Content-Type: ${contentType}" \
  -H "Authorization: AWS ${s3_access_key}:${signature_hash}" \
  https://${bucket}.s3.amazonaws.com/${file_to_upload}

I tested this script on CentOS-7 and CentOS-8.

curl aws s3