Skip to main content

Varicent ELT Assistant

SFTP import

Use the SFTP connector to connect your data to Varicent ELT.

Note

We auto-detect encoding, but if we can't detect a data set's file encoding, we will automatically use UTF-8 encoding.

You can also import flat files dynamically, such as CSV and Excel, with the filename and a wildcard (*) to import the file. With a timestamp, you can retrieve the last-modified file to import.

Importing data

  1. In Varicent ELT, select the Data module from the sidebar navigation pane.

  2. Click Import data.

  3. In the Third party sources section, click SFTP.

  4. Click New connection.

    Note

    If you already have an established SFTP connection, you will skip to the Select data page to import new data.

  5. On the Connect to your Model page, under Connector name, enter the name of the connection.

  6. Under Host, enter the host name for the connection.

  7. Under Port, enter the port number.

  8. Under Username, enter the user name to access the connection.

  9. Under Password, enter the password to access the connection.

  10. Optionally turn on the Signature verification key if you want to enable PGP signature verification on encrypted files, and in the Signature verification key field, and then paste the public PGP signing key.

    Note

    Ensure that the signature verification key is an RSA type.

  11. Click Next.

  12. On the Select data page, in the Static file section, under Directory, select the directory for the file that you want to import.

  13. Under File type, select the file type that you want to import.

  14. Under Delimiter, select the delimiter that you want to use from the following delimiters in the drop-down list:

    • Comma (,)

    • Pipe (|)

    • Semi-colon (;)

    • Space ( )

    • Tab ( )

    • Tilde (~)

  15. Under Quote, select the type of quotes that you want to use, such as double quotes (").

  16. Under Exclude excess data, enter the number of rows that you want to ignore at the beginning or end of your file.

  17. Under Header, check to designate the first unignored row as the column header. Leave unchecked to have column headers automatically generated.

  18. In the Dynamic file section, under File location, select the folder in the directory where the file is located.

  19. Under Match file name on *, type the name of the file, or enter a partial name with an asterisk * wildcard. For example, File* could retrieve all results with file in the name, such as File123, File456, and so on. Select the file that you want to import when it appears in the list.

  20. Under Select by, choose between Oldest modified data or Newest modified data.

  21. Under File type, select the file type that you want to import.

  22. Under Delimiter, select the delimiter that you want to use from the following delimiters in the drop-down field:

    • Comma (,)

    • Pipe (|)

    • Semi-colon (;)

    • Space ( )

    • Tab ( )

    • Tilde (~)

  23. Under Quote, select the type of quotes that you want to use, such as double quotes (").

  24. Under Exclude excess data, enter the number of rows that you want to ignore in the beginning or end of your file.

  25. Under Header, check to designate the first unignored row as the column header. Leave unchecked to have column headers automatically generated.

  26. Click Next.

  27. On the Choose data sets page, select which data sets you want to include.

  28. On the Import data page, review your data and select the data to obfuscate (if any). Click Next.

  29. On the Filter data page, optionally click Filter data to refine your data, or click Import data to continue.

    1. Filter the rows in your data by selecting the column to filter, the filter category, and the value or text.

    2. Click Import data to continue.

    Note

    For more information about how to configure data filters, see: Filter data.

  30. On the Review page, review the data sets that you want to import.

    Note

    You can also change the name of the data sets before you import the data. Click the pencil icon pencil_icon.png to edit the text field.

  31. Click Import data.

    The information is available on the My data page.

SFTP connector and Signature Verification Keys

When adding a new SFTP connection, a new optional checkbox, Signature Verification Key, is available on the Add New Connection page. Ensure that when you add a new signature verification key, the key is of the RSA type. When you provide the Signature Verification Key, it is securely stored, and Varicent ELT verifies the file signature before decrypting it with Varicent ELT's Private Key during import. If Select to decrypt file is not selected, signature verification is skipped.

If you want to use a Signature Verification Key in your current SFTP connector and currently don't have one, or if you want to modify an existing one, you can add or replace it in the Signature Verification Key section. When you are on the Edit Connection page for the SFTP connector, if you add or edit the key, and then Varicent ELT verifies the key.

When you edit an existing SFTP Connection, you can manage the signature key with three options:

  • Use Existing: Retains the stored key for verification.

  • Use New: Updates the stored key for future verification.

  • Remove: Deletes the signature key, skipping verification for subsequent imports.