Exporting a table

The following list explains what exporting a table means:

  • Saving data from a Xcalar table as one or multiple files to a location where other applications (or Xcalar) can access and use as a data source.
  • Saving the result of a batch dataflow to another table in the active worksheet.

You can export data from an entire table or from selected columns of a table.

NOTE: Columns of the Array or Object type cannot be exported.

Availability of the export feature

Xcalar in modeling mode cannot run a batch dataflow. As a result, it can only export a table to a file system. You cannot export the result of a batch dataflow if you use Xcalar in modeling mode.

Methods for exporting

You can export a table in one of two ways:

  • Manually export a table from the table drop-down menu.
  • Automatically export a table resulting from a batch dataflow. For more information about batch dataflows, see Using batch dataflows.

Where tables are exported by default

By default, tables are exported to an export target named Default. The path to this export target might vary from one Xcalar installation to another. To determine your default export target, follow these steps:

  1. Click the Export Targets icon (under the Datasets icon) in the Xcalar Design toolbar.
  2. In the Export Targets panel, click the icon for export target named Default.

Understanding user-defined export targets

Creating an export target is necessary if one or both of these conditions are true:

  • You want to save export files to a location other than the default location.
  • You want a table to be processed by a UDF when it is being exported.
IMPORTANT: The directory used as the export target must be a shared directory reachable by all Xcalar nodes. Also, it must be writable by Xcalar. For example, to use /nas/xcalar_exports as a target, make sure that it is shared by all nodes and Xcalar has write permission to it. Contact the Xcalar cluster administrator to confirm that these requirements are met.

If the directory specified as the target does not exist, Xcalar creates the directory. For example, if /nas/xcalar_exports does not exist, it is created automatically during the export, provided that Xcalar has write permission to /nas and all nodes have access to the path.

Understanding export target types

An export target can be one of the following types:

  • File system: Xcalar exports data from an active table or from a batch dataflow to a file system that Xcalar can write to. No UDF is run when the export takes place.
  • UDF: When Xcalar exports data from an active table or from a batch dataflow, it processes the data with a UDF. For the export target, you can specify the file system to which the data is exported. The path to this file system is passed into the UDF for processing. A UDF used as an export target is called an export UDF. For an example of coding and applying an export UDF, see Example of using a UDF when exporting.

How export files are named

If you manually export a table, a default name of the export file is provided in the EXPORT TABLES panel. If you run a batch dataflow to export the resultant table, the default export file name is shown below the exported table icon of the batch dataflow graph. (The exported table icon is the last table icon in the graph.)

By default, Xcalar Design displays an error if it detects a file name conflict when exporting a table. To avoid the name conflict, you must create a unique export file name. Information about creating or editing an export file name is included in Exporting a table from the table drop-down menu.

When manually exporting a table, you can specify in the EXPORT TABLE panel that the newly exported table overwrites the existing file or appends to the existing file.

Format of export files

By default, tables are exported to csv files. You can, however, process a table with a UDF to save it to a file in another format. For example, if you want to export a table to a file to be used by Excel, you can use a UDF to parse the table so that it can be saved in the Excel file format.

You must create an export target in advance if you want to use a UDF to process your table before saving it to an export file. Also, the UDF must be accessible to Xcalar Design. For information about UDFs, see Understanding UDFs.

Creating an export target

Follow these steps to create an export target:

  1. Create a file system that will be used as the export location. For example, create the directory /nas/xcalar_exports.

  2. Click to display two icons below it. Then click to display the Export Target panel and the Create Export Target window as illustrated in the following screenshot:
  3. In the Create Export Target window, the information about the default export target is displayed. To create a target, click in the Export Targets panel. The Create Export Target window now displays fields for you to define a new export target.
  4. In the Create Export Target window, specify the name of the target. You need to refer to the target name when you export a table in the future.

  5. Select the export target type.

  6. Enter the path of the export folder created in step 1. Export files will be saved to this location.

  7. If the export target type is UDF, follow these steps:

    1. Select the UDF module.
    2. Type the function name (if it is not already displayed), which must be main.
  8. Click ADD.

Displaying export target information

In the Export Targets panel, click an export target icon to display information about the target.

Deleting an export target

In the window where the export target information is displayed, click DELETE to delete the target.

Alternatively, right click an export target icon to display a pop-up menu, in which you can click Delete Target.

NOTE: You cannot edit an export target. To change the name or definition of an export target, delete and re-create it.

Exporting a table from the table drop-down menu

Follow these steps to export a table:

  1. In your worksheet, locate the table that you want to export. Click the table title bar to display the table drop-down menu.
  2. Select Export table in the drop-down menu. The EXPORT TABLE panel is displayed.
  3. In the EXPORT TABLE panel, fill out the following information:

    • Table to Export: If the table displayed is not the one you want to export, use the Down arrow to display a list of tables in the active worksheet and select the table from the list.
    • New Folder Name: Specify a unique name for the export file.
    • Export Location: This is the export target to be used for exporting the table. Use the Down arrow to display a list of targets that you have created. Select the export target named Default if you want to use the default export target. To determine the location of the default export target, see Where tables are exported by default.
    • Columns to export: By default, all columns are exported. You can specify that only particular columns are included in the exported file.
  4. (Optional) Click Preserve order (slow) if you want the rows in the export file to be sorted in the same way as the table in your worksheet. Sorting the rows causes the export to take a longer time. This option is available if both of these conditions are true:

    • The table is exported without being processed by a UDF.
    • The table rows are already sorted.
  5. (Optional) If you want to further customize the way in which the table is saved to the export file. see Advanced options for exporting a table.
  6. Click EXPORT.

Advanced options for exporting a table

The advanced options in the EXPORT TABLE panel enables you to specify different ways to export a table:

Field Delimiter and Record Delimiter

You can specify the field delimiter and record delimiter in the same way as you specify these settings when importing a data source.

File

By default, the table is exported to a single file. You can specify that the table be exported to multiple files. You cannot, however, specify the exact number of files to be created. Xcalar Design automatically decides the number of export files.

NOTE: Specifying Multiple Files does not always result in multiple export files. It is possible that the table is exported to a single file even if this option is selected.

Header

By default, the column headers are saved to each export file. You can specify that the headers be saved to a separate file, in which case the data rows are saved to a file or files without the headers.

Overwrite

You can select one of the three ways of overwriting existing export files:

  • Do Not Overwrite: By default, the table is exported to the file named in the export target. If a table has been saved to the file, Xcalar Design does not overwrite the file. It displays an error message that tells you to create a new export file name.
  • Overwrite Existing: You can choose to overwrite the existing file. If the export file named in the export target already contains a table previously exported to the file, the contents of the export file is overwritten.
    NOTE: This option overwrites the contents of the entire directory to which a table was previously exported, regardless of how many export files are created as a result of exporting the current table. For example, a directory named /mnt/xcalar/export/airlines contains multiple export files, which are named airlines-001.csv and airlines-002.csv. If you use airlines as the export file name to export the current table as a single file, the /mnt/xcalar/export/airlines directory will contain the export file named airlines.csv. The files named airlines-001.csv and airlines-002.csv no longer exist.
  • Append To Existing: You can add the newly created export file to an export location that already contains export files. How the appending is done depends on whether the table is exported as a single file or multiple files, as illustrated by the following example.

    EXAMPLE: If the export target already contains airlines-001.csv and airlines-002.csv, exporting a table as multiple files creates airlines-003.csv and airlines-004.csv in this directory. If the export target already contains airlines.csv, exporting a table as a single file appends the exported table's rows to airlines.csv.
    NOTE: You cannot append a table being exported as a single file to multiple existing export files. Similarly, you cannot append a table being exported as multiple files to a single existing export file.

Exporting a table when running a batch dataflow

As the last step of running a batch dataflow, Xcalar Design exports a table (or selected columns of a table). The procedure for running a batch dataflow is explained in Using batch dataflows.

For more information about the purpose of batch dataflows and how to create batch dataflows, see Understanding and creating batch dataflows.

Go to top