After data is ingested to a Data Lakehouse, what happens if the data is structurally modified by a process outside of Infoworks and the metastore records are changed?

Sep 23, 2025 · The getting started chapter will walk you through the following Infoworks processes: now 6.2.1. Kubernetes-Based Deployment for AKS Configuring and Managing Environments Navigating …

Infoworks agile data engineering platform automates the creation and operation of big data workflows from source to consumption, both on premise and in the cloud.

This service listens to the changes on the source cluster HDFS, and writes the changes in the Infoworks metastore. The incremental replicator will then replicate the changes after reading it from the metastore.

To modify the Precision and Scale for a column in the table in the Infoworks source, ensure the source and target data are the same and there are no corrupt/modified values in the target Hive tables.

Navigate to Admin section of Infoworks UI, and click on external scripts, which shows up pipeline extensions page. Click on Add an extension,and configure it as follows:

Infoworks will partition the column range into slices based on the number of connections allowed. Example: A table with a user_id column ranging from 1 to 1000, split into 10 ranges, will result in 10 …

Step 4: Configure a Custom Role on Infoworks If the default role is not sufficient, you can explicitly pass a different role in the platform configuration: Go to your Snowflake Data Environment settings in the …

Post installation, Infoworks can be configured and connected to the required environment from the user interface (UI). This environment can be defined from preset templates, or can also be defined as a …

When testing a connection to a data source from Infoworks, users may observe that the Test Connection fails, even though the network settings on the Infoworks AKS cluster appear to be correct.