Version: Next

DataHub Releases

Summary

Released on 2024-05-23 by @david-leifker.

NEW: Business Attributes: Business Attributes are used to standardize and manage data elements across multiple domains, projects, and applications. By linking dataset attributes to Business Attributes, organizations ensure uniformity and ease of updates, as changes made to a Business Attribute are automatically propagated across all linked datasets. #9863
Improved UI for Dataset Properties: Added collapse functionality for long dataset properties, making it easier to navigate and view relevant information. [#10203](https://github.com/datahub-project/datahub/pull/10203)
Pagination for Ingestion Tasks Listing: Added pagination to the tasks listing page, making it easier to manage and navigate through tasks. [#10293](https://github.com/datahub-project/datahub/pull/10293)
Rich Text Support for Form Descriptions: Added support for rich text in form descriptions, enhancing the user experience. [#104](https://github.com/datahub-project/datahub/pull/104)2 5
New Analytics Charts: Added charts in the Analytics tab to identify Top Users and New Users. #10344
Enhanced search functionality with customizable autocomplete configuration. [#10426](https://github.com/datahub-project/datahub/pull/10426)

Unified CI Workflow Updates: Improved CI build with unified workflow updates and disk space cleanup, making the build process more efficient. [#10353](https://github.com/datahub-project/datahub/pull/10353)
Improved Logging for GraphQL Requests: Enhanced logging for GraphQL requests, providing better insights and debugging capabilities. [#10404](https://github.com/datahub-project/datahub/pull/10404)
Enhanced Documentation for Lineage Feature Guide: Updated documentation for the lineage feature guide, making it easier to understand and implement. [#10401](https://github.com/datahub-project/datahub/pull/10401)
Improved Documentation for SchemaField.label: Updated documentation for SchemaField.label, providing clearer guidance for developers. [#10251](https://github.com/datahub-project/datahub/pull/10251)
Enhanced CI with Docker Image Publishing: Added Docker image publishing capabilities to the CI workflow, streamlining the deployment process. [#10193](https://github.com/datahub-project/datahub/pull/10193)
Redesigned Docs Site Feedback Button: Improved the design of the feedback button in the documentation, making it more user-friendly. [#10182](https://github.com/datahub-project/datahub/pull/10182)

Improved Data Profiling by early filtering of tables, correctly computing sample row counts, and combining unique count queries per table. #10378, #10319, #10322
Airflow: Introduced support for BigQueryInsertJobOperator. #10452
BigQuery: Added support for Table Clones and incremental column-level lineage.
Snowflake: Improved reporting for usage aggregation and handled lineage errors; Improved ingestion performance with system sampling on very large tables. #10279, #10430
Glue: Introduced support for delta schemas. [#10299](https://github.com/datahub-project/datahub/pull/10299)
Redshift: Improved usage extraction by filtering out system queries. #10247
Mode: Enhanced ingestion for Mode by adding dashboards into containers, improving data visualization and management. [#10563](https://github.com/datahub-project/datahub/pull/10563)
PowerBI: Added support to automatically extract table lineage between PowerBI and Databricks. [#10416](https://github.com/datahub-project/datahub/pull/10416)
dbt: Improved dbt ingestion by handling complex SQL and enhancing documentation, providing better data management and insights. [#10323](https://github.com/datahub-project/datahub/pull/10323)
NiFi: Enhanced ingestion for NiFi with process group as browse path and incremental lineage, improving data organization and tracking. [#10202](https://github.com/datahub-project/datahub/pull/10202)
Incubating Sigma and CockroachDB sources. #10037, #10226

DynamoDB Connector: aws_region is now a required configuration. The connector will no longer loop through all AWS regions; instead, it will only use the region passed into the recipe configuration. [#10419](https://github.com/datahub-project/datahub/pull/10419)
Custom Validators and Mutators: Dropped a previously required constructor. [#10389](https://github.com/datahub-project/datahub/pull/10389)
FabricType RVW: Added as a new FabricType. No rollbacks allowed once metadata with this fabric type is added without manual cleanups in databases. [#10472](https://github.com/datahub-project/datahub/pull/10472)

For full details on breaking changes, please refer to the updating DataHub documentation.