AWS - Redshift - Load S3 Data

Getting your Trinity Audio player ready...

Some time ago I did a job that theoretically seemed simple, extracting data from a transactional database and sending it to Redshift for analysis.

Of course, after hitting my head for a few minutes I understood that importing data directly into Redshift would be at least troubled and unstable.

Doing an analysis of the options I saw that AWS provided a method very similar to that of SQL Server to import files directly into the database, but of course, instead of doing this through a server, it is possible to do this through S3.

The simplest way is basically:

copy destination_table

from 'S3: //bucket/file'

iam_role 'arn:aws:iam: :01234567890:rolle/my Redshift Rule'

If the file is very large and has been split it has to end with an incremental numeral 1 2 3 4...

If the file is compressed, the COPY command must be incremented with GZIP.

For more information, see the AWS link below:

to monitor this import you can use

Leave a Reply

Your email address will not be published. Required fields are marked *