Releases: Azure/azure-kusto-spark
Releases · Azure/azure-kusto-spark
v2.4_3.1.4
** Bug fix **
- Fix reading fallback from Single read mode to Distributed to occur only in case of exceptions that state "Query execution has exceeded the allowed limits" in which cases we expect to succeed in Distributed.
- Fix kusto extension for streaming
- Fix retry exceptions logic - could throw in certain exceptions types. Exception type for retrying of commands was wrong (was hiding real error)
- Fix #260. Creates an identity mapping in CSV in the order of columns when create table and GenerateCsvMapping is specified
v3.0_3.1.3
** Bug fix **
- Fix decimals to new scales 38,18 the default in Spark
v2.4_3.1.3
** Bug fix **
- Fix decimals to new scales 38,18 the default in Spark
v2.4_3.1.2
Bug Fix
- Added retry mechanism for every request to the service
New feature
- Added Debug (verbose) logging on polling progress for both ingestion and async operations (read distributed and move extents)
- Changed polling time to increase exponentially only after a minute according to com.microsoft.kusto.spark.utils.KustoDataSourceUtils.{ReadInitialMaxWaitTime,ReadMaxWaitTime,WriteInitialMaxWaitTime,WriteMaxWaitTime}
v3.0_3.1.2
Bug Fix
- Added retry mechanism for every request to the service
New feature
- Added Debug (verbose) logging on polling progress for both ingestion and async operations (read distributed and move extents)
- Changed polling time to increase exponentially only after a minute according to com.microsoft.kusto.spark.utils.KustoDataSourceUtils.{ReadInitialMaxWaitTime,ReadMaxWaitTime,WriteInitialMaxWaitTime,WriteMaxWaitTime}
v3.0_3.1.1
** Bug fix **
- Branch 2.4 failed for Task serialization
v2.4_3.1.1
** Bug fix **
- Branch 2.4 failed for Task serialization
v2.4_3.1.0-known-issue
** known-issue **
- new option KUSTO_POLLING_ON_DRIVER is failing with default value (false)
** Breaking change **
- KUSTO_POLLING_ON_DRIVER is set to false by default - its recommended to set this to true once in production!
** New Feature **
- Batching policy can now be set over a table and will take effect on temporary table
- New option : An existing temporary table can be given as a parameter
- New write mode - Queued mode - good for high velocity cases in which the user doesn't need the connector to handle every failure and success - this is similar to Kusto queued ingestion and the user can still use the Queue status report method for queued ingestion:https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/netfx/kusto-ingest-client-status#ingestion-status-in-azure-queue
- New option: ingestion uri - allows to provide the ingestion uri if it can't be deduced from the query endpoint
** Bug Fix **
- If no read mode provided and count estimation failed, we could estimate the query to be empty
- Fix conflict with nimbusds versions
v3.0_3.1.0
** Breaking change **
- KUSTO_POLLING_ON_DRIVER is set to false by default - its recommended to set this to true once in production!
** New Feature **
- Batching policy can now be set over a table and will take effect on temporary table
- New option : An existing temporary table can be given as a parameter
- New write mode - Queued mode - good for high velocity cases in which the user doesn't need the connector to handle every failure and success - this is similar to Kusto queued ingestion and the user can still use the Queue status report method for queued ingestion:https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/netfx/kusto-ingest-client-status#ingestion-status-in-azure-queue
- New option: ingestion uri - allows to provide the ingestion uri if it can't be deduced from the query endpoint
** Bug Fix **
- If no read mode provided and count estimation failed, we could estimate the query to be empty
- Fix conflict with nimbusds versions
v2.4_3.0.2
** Bug Fix from v3.0.1 **
- Fix NPE on move extents when target is materialized views
- Don't swallow timeout exception on queued ingestion
** New Feature **
- New option -KUSTO_POLLING_ON_DRIVER - Polling and Move extents as spark job - currently disabled by default - next version will be enabled by default