Use the New Big Data Connection dialog box

The New Big Data Connection dialog box is a single, guided experience that allows you to connect to your datasets and flexibly set properties such as time and geometry. The big data connection dialog box simplifies and expedites data preparation by removing time-consuming workflows, such as calculating geometry, formatting fields, or combining datasets. The big data connection dialog box allows you to do the following:

  • Connect to your data and create a big data connection.
  • Edit your big data connection dataset.
  • Preview your datasets.

To access the New Big Data Connection dialog box, on the Insert tab, in the Project group, click the Connections drop-down arrow Add Database and click New Big Data Connection Big Data File Share.

There are three steps when creating a big data connection:

  1. Connect to your big data connection.
  2. Configure your big data connection datasets.
  3. Review your big data connection.

Connect

Connecting to a big data connection requires a name for your connection and a path to a folder of datasets. Once you have completed the big data connection name and source folder, click Create connection to establish a connection to the source folder. This returns successful and failed datasets. Review the datasets to ensure that the datasets look as expected and address any issues. If the datasets look as expected, click Next.

Learn more about why a big data connection dataset wasn't successful

Configure

When configuring your big data connection, you can complete the following operations on any dataset:

  • Duplicate—Duplicate the reference to a dataset, so you can register it in multiple ways.
  • Delete—Remove the reference to the dataset.
  • Edit properties—Modify the time, geometry, fields, file settings, and filters for a dataset.
  • Preview—See a preview of the registered dataset.

You can access these options by selecting a dataset, clicking one of the buttons, or right-clicking and choosing from the options. Editing and previewing are described in more detail below.

When you create a big data connection, it's important to make sure it's correctly referenced. To make sure it's working as expected, use the preview option to see a sample of the data and look at the configured properties to make sure they match your expectations.

Edit properties

Editing properties allows you to modify how a big data connection is represented. You can modify five different properties based on the source data type:

  • Fields—Modify the field names and types for delimited files.
  • Time—Set how date and time are represented by specifying date and time fields, formats, and the time zone.
  • Geometry—Set how geometry is represented by specifying the geometry type, fields used to represent geometry, and the spatial reference.
  • File—Set the file format for delimited files.
  • Filter—Apply a filter to use a subset of the dataset.

Preview

The preview shows the first 10 records of a dataset. Preview a dataset after it has been discovered or after the properties have been edited. Verify that fields and values look as expected.

Note:

The preview doesn't show the geometry registration. To verify that the geometry and time look as expected, use the Describe Dataset tool on your big data connection.

Review

Review the output location of your big data connection, the source folder you connected to, and the datasets that were discovered. Return to the Configure pane to make any changes.

Finish

After you configure your big data connection, click the Finish button to finish. The big data connection file (.bdc) is created in your project's home folder.