Skip to content

Releases: Azure/azure-kusto-spark

v2.4_3.1.4

02 Nov 11:01
Compare
Choose a tag to compare

** Bug fix **

  • Fix reading fallback from Single read mode to Distributed to occur only in case of exceptions that state "Query execution has exceeded the allowed limits" in which cases we expect to succeed in Distributed.
  • Fix kusto extension for streaming
  • Fix retry exceptions logic - could throw in certain exceptions types. Exception type for retrying of commands was wrong (was hiding real error)
  • Fix #260. Creates an identity mapping in CSV in the order of columns when create table and GenerateCsvMapping is specified

v3.0_3.1.3

15 Sep 12:48
914627c
Compare
Choose a tag to compare

** Bug fix **

  • Fix decimals to new scales 38,18 the default in Spark

v2.4_3.1.3

15 Sep 12:54
Compare
Choose a tag to compare

** Bug fix **

  • Fix decimals to new scales 38,18 the default in Spark

v2.4_3.1.2

25 Aug 12:02
Compare
Choose a tag to compare

Bug Fix

  • Added retry mechanism for every request to the service

New feature

  • Added Debug (verbose) logging on polling progress for both ingestion and async operations (read distributed and move extents)
  • Changed polling time to increase exponentially only after a minute according to com.microsoft.kusto.spark.utils.KustoDataSourceUtils.{ReadInitialMaxWaitTime,ReadMaxWaitTime,WriteInitialMaxWaitTime,WriteMaxWaitTime}

v3.0_3.1.2

25 Aug 08:25
b57d073
Compare
Choose a tag to compare

Bug Fix

  • Added retry mechanism for every request to the service

New feature

  • Added Debug (verbose) logging on polling progress for both ingestion and async operations (read distributed and move extents)
  • Changed polling time to increase exponentially only after a minute according to com.microsoft.kusto.spark.utils.KustoDataSourceUtils.{ReadInitialMaxWaitTime,ReadMaxWaitTime,WriteInitialMaxWaitTime,WriteMaxWaitTime}

v3.0_3.1.1

15 Aug 12:11
Compare
Choose a tag to compare

** Bug fix **

  • Branch 2.4 failed for Task serialization

v2.4_3.1.1

15 Aug 17:16
Compare
Choose a tag to compare

** Bug fix **

  • Branch 2.4 failed for Task serialization

v2.4_3.1.0-known-issue

11 Aug 09:22
Compare
Choose a tag to compare

** known-issue **

  • new option KUSTO_POLLING_ON_DRIVER is failing with default value (false)

** Breaking change **

  • KUSTO_POLLING_ON_DRIVER is set to false by default - its recommended to set this to true once in production!

** New Feature **

  • Batching policy can now be set over a table and will take effect on temporary table
  • New option : An existing temporary table can be given as a parameter
  • New write mode - Queued mode - good for high velocity cases in which the user doesn't need the connector to handle every failure and success - this is similar to Kusto queued ingestion and the user can still use the Queue status report method for queued ingestion:https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/netfx/kusto-ingest-client-status#ingestion-status-in-azure-queue
  • New option: ingestion uri - allows to provide the ingestion uri if it can't be deduced from the query endpoint

** Bug Fix **

  • If no read mode provided and count estimation failed, we could estimate the query to be empty
  • Fix conflict with nimbusds versions

v3.0_3.1.0

09 Aug 12:58
774ffb8
Compare
Choose a tag to compare

** Breaking change **

  • KUSTO_POLLING_ON_DRIVER is set to false by default - its recommended to set this to true once in production!

** New Feature **

  • Batching policy can now be set over a table and will take effect on temporary table
  • New option : An existing temporary table can be given as a parameter
  • New write mode - Queued mode - good for high velocity cases in which the user doesn't need the connector to handle every failure and success - this is similar to Kusto queued ingestion and the user can still use the Queue status report method for queued ingestion:https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/netfx/kusto-ingest-client-status#ingestion-status-in-azure-queue
  • New option: ingestion uri - allows to provide the ingestion uri if it can't be deduced from the query endpoint

** Bug Fix **

  • If no read mode provided and count estimation failed, we could estimate the query to be empty
  • Fix conflict with nimbusds versions

v2.4_3.0.2

31 Jul 13:38
Compare
Choose a tag to compare

** Bug Fix from v3.0.1 **

  • Fix NPE on move extents when target is materialized views
  • Don't swallow timeout exception on queued ingestion

** New Feature **

  • New option -KUSTO_POLLING_ON_DRIVER - Polling and Move extents as spark job - currently disabled by default - next version will be enabled by default