Please enter your information to subscribe to the Microsoft Fabric Blog.

Thanks for subscribing to the Microsoft Fabric Blog. You’ll receive email notifications when we publish new posts.

Microsoft Fabric Updates Blog

  1. Updates
  2. Apache Spark

Introducing High Concurrency Mode for Notebooks in Pipelines for Fabric Spark

We’re excited to introduce high concurrency mode for notebooks in pipelines, bringing session sharing to one of the most popular orchestration mechanisms for enterprise data ingestion and transformation. Notebooks will now automatically be packed into an active high concurrency session without compromising performance or security, while paying for a single session. Key Benefits: Why Use … Continue reading “Introducing High Concurrency Mode for Notebooks in Pipelines for Fabric Spark”

Announcing the Fabric Apache Spark Diagnostic Emitter: Collect Logs and Metrics

Fabric Apache Spark Diagnostic Emitter for Logs and Metrics is now in public preview. This new feature allows Apache Spark users to collect Spark logs, job events, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. It provides robust support for monitoring … Continue reading “Announcing the Fabric Apache Spark Diagnostic Emitter: Collect Logs and Metrics”

Organizing your tables with lakehouse schemas and more (Public Preview)

We’re thrilled to introduce a new feature in Fabric: Lakehouse schemas. This feature lets users arrange their lakehouse tables into a folder-like structure, enhancing data discovery and more. Many users will be familiar with schemas in Fabric using Data Warehouse, and we are bringing aligned capabilities into Lakehouse. Schemas created in your lakehouses will also … Continue reading “Organizing your tables with lakehouse schemas and more (Public Preview)”

Advanced Time Series Anomaly Detector in Fabric

Anomaly Detector, one of Azure AI services, enables you to monitor and detect anomalies in your time series data. This service is based on advanced algorithms, SR-CNN for univariate analysis and MTAD-GAT for multivariate analysis and is being retired by October 2026. In this blog post we will lay out a migration strategy to Microsoft Fabric, allowing … Continue reading “Advanced Time Series Anomaly Detector in Fabric”

Building a Custom Sparklens JAR for Microsoft Fabric

Problem Statement In the previous blog on Profiling Microsoft Fabric Spark Notebooks with Sparklens, we covered how to run Sparklens to profile and tune the performance of your spark notebooks in Microsoft Fabric. In that blog, we used a custom Sparklens JAR. The Sparklens JARs available in the Maven Central repo supports only the Spark … Continue reading “Building a Custom Sparklens JAR for Microsoft Fabric”