Key details for this exam, checked against the published exam outline
Each question shows the correct answer and an explanation of why it is right
A data engineer is streaming data from Kafka and requires:
Minimal latency
Exactly-once processing guarantees
Which trigger mode should be used?
Exactly-once guarantees in Spark Structured Streaming require micro-batch mode (default), not continuous mode.
Continuous mode (.trigger(continuous=...)) only supports at-least-once semantics and lacks full fault-tolerance.
trigger(availableNow=True) is a batch-style trigger, not suited for low-latency streaming.
So:
Option A uses micro-batching with a tight trigger interval minimal latency + exactly-once guarantee.
Final Answer: A
A data analyst wants to add a column date derived from a timestamp column.
Options:
f.to_date() converts a timestamp or string to a DateType.
Ideal for extracting the date component (year-month-day) from a full timestamp.
Example:
from pyspark.sql.functions import to_date
dates_df.withColumn('date', to_date('timestamp'))
A Spark DataFrame df is cached using the MEMORY_AND_DISK storage level, but the DataFrame is too large to fit entirely in memory.
What is the likely behavior when Spark runs out of memory to store the DataFrame?
When using the MEMORY_AND_DISK storage level, Spark attempts to cache as much of the DataFrame in memory as possible. If the DataFrame does not fit entirely in memory, Spark will store the remaining partitions on disk. This allows processing to continue, albeit with a performance overhead due to disk I/O.
As per the Spark documentation:
'MEMORY_AND_DISK: It stores partitions that do not fit in memory on disk and keeps the rest in memory. This can be useful when working with datasets that are larger than the available memory.'
--- Perficient Blogs: Spark - StorageLevel
This behavior ensures that Spark can handle datasets larger than the available memory by spilling excess data to disk, thus preventing job failures due to memory constraints.
A data engineer is working on the DataFrame:
(Referring to the table image: it has columns Id, Name, count, and timestamp.)
Which code fragment should the engineer use to extract the unique values in the Name column into an alphabetically ordered list?
To extract unique values from a column and sort them alphabetically:
distinct() is required to remove duplicate values.
orderBy() is needed to sort the results alphabetically (ascending by default).
Correct code:
df.select('Name').distinct().orderBy(df['Name'])
This is directly aligned with standard DataFrame API usage in PySpark, as documented in the official Databricks Spark APIs. Option A is incorrect because it may not remove duplicates. Option C omits sorting. Option D sorts in descending order, which doesn't meet the requirement for alphabetical (ascending) order.
A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference in event_timestamp. The engineer adds:
dropDuplicatesWithinWatermark("event_timestamp", "30 minutes")
What is the result?
The method dropDuplicatesWithinWatermark() in Structured Streaming drops duplicate records based on a specified column and watermark window. The watermark defines the threshold for how late data is considered valid.
From the Spark documentation:
'dropDuplicatesWithinWatermark removes duplicates that occur within the event-time watermark window.'
In this case, Spark will retain the first occurrence and drop subsequent records within the 30-minute watermark window.
Final Answer: B
135 questions covering all exam domains, starting from $20
Exam domains verified against: Official Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam guide, last checked August 2026.
Learn how Spark driver, executors, cluster manager, and SparkContext work together to distribute work across a cluster. Study execution modes, the execution hierarchy, fault tolerance, and lazy evaluation in real Spark applications.
Work with structured data through DataFrames and Spark SQL syntax. Practice aggregations, joins, transformations, and understand how the Catalyst optimizer executes your queries.
Build ETL workflows using the DataFrame and Dataset APIs to handle structured and semi-structured data. Apply column operations, row filtering, aggregations, and schema manipulation in real data pipelines.
Set up continuous data ingestion with Spark Structured Streaming. Master stateful and stateless operations, watermarking, output modes, checkpointing, and fault tolerance in streaming pipelines.
Understand how Spark Connect enables remote communication and execution of Spark applications. Learn deployment patterns that enhance performance and scalability in distributed systems.
Use the Pandas API on Apache Spark to run familiar Pandas operations at scale. Execute large-scale data analytics by combining Python simplicity with Spark's distributed computing.
Diagnose and debug Spark applications by inspecting execution plans and logs. Tune memory, parallelism, and configuration parameters to resolve performance bottlenecks.
Sample question from this domain above: Q3
Common questions about the exam itself
