Using UDFs

This section describes the tasks related to UDFs (user-defined functions). To see examples of UDFs, see User-defined function (UDF) examples.

Creating a UDF in Xcalar Design

UDFs are global. UDFs created by one Xcalar Design user can be used and edited by all other Xcalar Design users.

You define one or multiple UDFs in a Python module. You can create a module in these ways:

  • Type the code directly in Xcalar Design.
  • Upload from your computer.

Both methods require that you display the User Defined Function panel.

Displaying the UDF panel

Click to display the User Defined Function panel. The following screenshot shows the location of the UDF icon.

The following screenshot shows the User Defined Function panel.

Typing the code directly

Follow these steps to write the UDF in the panel and then save it:

  1. Select the Editor tab if it is not selected already.
  2. Click the drop-down arrow to display a list of existing modules. Select one to use it as a template. Alternatively, select Blank function if you want to type in an empty file.
  3. Type the code in the panel.
  4. In the module name field, type the module name.

    NOTE: Module names are case insensitive. If you type letters in uppercase, Xcalar Design displays them in lowercase when listing the modules. See UDF naming conventions for information about name requirements.
  5. Click ADD. If a module with the same name already exists, a dialog box asks whether you want to overwrite the existing module. If so, click CONFIRM. Otherwise, click CANCEL and then rename the module.

TIP: When editing a UDF, you can use the keyboard shortcut appropriate for your operating system to search for or replace a string. For example, if you run Xcalar Design in a browser on Microsoft Windows, press the Ctrl and F keys to search for a string, and the Ctrl, Shift, and F keys to replace a string.

Uploading files

Follow these steps to upload a file as a UDF module from your computer to your cluster:

  1. Select the Editor tab if it is not selected already.
  2. Click Browse File, which enables you to navigate to the folder on your computer that contains the file. Then select the file. The contents of the selected file are displayed in the User Defined Function panel.
  3. In the module name field, the name of the file is automatically displayed. Change the name if necessary.
  4. NOTE: If the original filename does not follow the naming conventions, an error message notifies you that the name is invalid. See UDF naming conventions for name requirements.
  5. Click ADD. If a module with the same name already exists, a dialog box asks whether you want to overwrite the existing module. If so, click CONFIRM. Otherwise, click CANCEL and then rename the module.

    NOTE: The icon next to the ADD button indicates whether the module will be added as a UDF or an XPU app. Uploading an app is not yet supported; you can only upload a UDF, which is represented by .

About Python code parsing

When you try to add a UDF, it is parsed by the Python interpreter. If there are parse errors, they are reported to you and the UDF is not added.

Downloading a module

Follow these steps to download a UDF module from your cluster to your computer:

  1. Select the Manager tab in the User Defined Function panel. A list of modules is displayed.
  2. Click to download the desired module to your computer.

Editing a UDF

IMPORTANT: Because UDFs are global and accessible to all Xcalar Design users, remember to notify other users of the changes so that they are aware of the revision. Also, changing the code in a UDF might have significant impact on both workbook re-activation and batch dataflows. Make sure you understand the effects of changing a particular UDF before proceeding.

Follow these steps to edit an existing UDF:

  1. Select the Manager tab in the User Defined Function panel. A list of modules is displayed.
  2. Edit the module in one of two ways:
    • Click to edit the desired module. Then edit the code in the User Defined Function panel. Add the edited module to Xcalar without changing its name.
    • Click to download the desired module to your computer. Then modify the module by using an editor on your computer. Upload the file after you finish editing.

Effect of changing a UDF on workbook re-activation

When you re-activate a workbook, Xcalar must re-create each table in the workbook according to the information contained in the table's dataflow. If a dataflow contains a UDF that has been changed since the table was created, the table displayed in the re-activated workbook might contain values different from the original ones.

Effects of changing a UDF on a batch dataflow

If a batch dataflow contains a UDF and that UDF has been changed, Xcalar uses the modified UDF, not the original UDF when the batch dataflow was created, during the execution of the batch dataflow.

Deleting a module

IMPORTANT: Be sure you understand the effects of a UDF deletion before you proceed. You cannot undo a UDF deletion. For more information, see Effects of UDF deletion on workbooks and batch dataflows.

Follow these steps to delete a UDF module:

  1. Select the Manager tab in the User Defined Function panel. A list of modules is displayed.
  2. Click to delete the module.

Effects of UDF deletion on workbooks and batch dataflows

The following list describes what happens after you remove a function from a UDF module or delete a UDF module from the cluster. Because UDFs are shared by all users on the cluster, all users who have applied the UDF are affected in the same way.

  • When Xcalar Design tries to re-activate a workbook (for example, after Xcalar Compute Engine is restarted or when you switch from one workbook to another):

    If the workbook contains a temporary or active table that is dependent on the UDF, Xcalar Design displays an error message reporting that the UDF cannot be found. As a result, the workbook cannot be activated until you re-create the UDF with the same name.

    A table is considered to be dependent on the UDF if one of these conditions applies:

    • The table has a column created by the UDF.
    • The table is a descendent of a dropped table that contains a column created by the UDF.
  • When you try to run a batch dataflow:

    If the batch dataflow contains the deleted UDF, Xcalar Design displays an error message, as shown in the following screenshot. The batch dataflow cannot be run until you re-create the UDF with the same name.

  • When you try to create a batch dataflow:

    If the dataflow contains the deleted UDF, Xcalar Design displays an error message. You cannot create a batch dataflow from a dataflow with a missing UDF.

Go to top