Skip to main content

Varicent ELT Help Center

Amazon S3 import

Abstract

To import your data, connect your Varicent ELT Amazon S3 accounts.

Connect your Varicent ELT and Amazon S3 accounts to import your data. Amazon S3 accepts the following file types:

  • CSV

  • Excel

  • Parquet

  • .txt

Importing data

  1. In Varicent ELT, select the Data module from the sidebar navigation pane.

  2. Click Import data.

  3. In the Third party sources section, click Amazon S3.

  4. Click New connection.

  5. On the Connect to your Data page, enter the following information to connect:

    Table 13. Amazon S3

    Field

    Description

    Connector name

    The name of the connection.

    Account ID

    The ID of the AWS account that you want to use.

    Bucket Name

    The name of the AWS bucket in your Amazon S3 instance that you connect to.

    Region

    The region of your Amazon S3 bucket.



  6. Click Next.

  7. On the next Connect to your Data page, follow the on-screen steps to set up your account.

    1. Follow the on-screen steps to create a new policy in your AWS account.

      Note

      The policy body is pre-populated with the information that you had entered in the previous step. Use the copy button to copy the code.

    2. Follow the on-screen steps to create a new role in your AWS account.

      Note

      The account ID value is based on the environment that you are in.

    3. Create a unique name in the Role Name field.

    4. Click Next.

  8. On the Select data page, enter the following information to import your data:

    Field

    Description

    Directory

    Select the directory for the file that you want to import. For example, if you want to select Parquet files, ensure that you select the folder containing your Parquet files.

    File type

    Select the file type that you want to import, such as CSV, Excel, .txt or Parquet.

    Caution

    If you select a directory with Parquet files, ensure that you select the Parquet file type. If you select another type of file, the import will fail.

    Note

    If you select the Parquet file option, you won't have to specify the Delimeter and other configuration options, just follow the import wizard.

    Delimiter

    Select the delimiter that you want to use from the following delimiters in the drop-down field:

    • Comma (,)

    • Pipe (|)

    • Semi-colon (;)

    • Space ( )

    • Tab ( )

    • Tilde (~)

    Quote

    Select the type of quotes that you want to use, such as double quotes (").

    Exclude excess data

    Enter the number of rows that you want to ignore in the beginning or end of your file.

    Header

    Check to designate the first unignored row as the column header. Leave unchecked to have column headers automatically generated.

  9. Click Next.

  10. On the Import data page, review your data and select the data to obfuscate (if any). Click Next.

  11. On the Filter data page, optionally click Filter data to refine your data, or click Import data to continue.

    1. Filter the rows in your data by selecting the column to filter, the filter category and the value or text.

    2. Click Import data to continue.

    Note

    For more information about how to configure data filters, see: Filter data.

  12. On the Review page, review the data sets that you are importing.

  13. Click Import data.

    The data is available on the My Data page.