[Aug-2023] Pass Databricks-Certified-Professional-Data-Engineer Exam in First Attempt Updated Databricks-Certified-Professional-Data-Engineer Exam Questions [Q106-Q124]

4.2/5 - (4 votes)

[Aug-2023] Pass Databricks-Certified-Professional-Data-Engineer Exam in First Attempt Updated Databricks-Certified-Professional-Data-Engineer Exam Questions

Databricks Certification Dumps Databricks-Certified-Professional-Data-Engineer Exam for Full Questions – Exam Study Guide

QUESTION 106
You have written a notebook to generate a summary data set for reporting, Notebook was scheduled using the job cluster, but you realized it takes an average of 8 minutes to start the cluster, what feature can be used to start the cluster in a timely fashion?

Setup an additional job to run ahead of the actual job so the cluster is running second job starts

Use the Databricks cluster pools feature to reduce the startup time

Use Databricks Premium edition instead of Databricks standard edition

Pin the cluster in the cluster UI page so it is always available to the jobs

Disable auto termination so the cluster is always running

QUESTION 107
A dataset has been defined using Delta Live Tables and includes an expectations clause: CON-STRAINT valid_timestamp EXPECT (timestamp > ‘2020-01-01’) ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?

Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.

Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.

Records that violate the expectation cause the job to fail.

Records that violate the expectation are added to the target dataset and flagged as in-valid in a field added to the target dataset.

Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.

QUESTION 108
The below spark command is looking to create a summary table based customerId and the number of times the customerId is present in the event_log delta table and write a one-time micro-batch to a summary table, fill in the blanks to complete the query.
1.spark._________
2. .format(“delta”)
3. .table(“events_log”)
4. .groupBy(“customerId”)
5. .count()
6. ._______
7. .format(“delta”)
8. .outputMode(“complete”)
9. .option(“checkpointLocation”, “/tmp/delta/eventsByCustomer/_checkpoints/”)
10. .trigger(______)
11. .table(“target_table”)

writeStream, readStream, once

readStream, writeStream, once

writeStream, processingTime = once

writeStream, readStream, once = True

readStream, writeStream, once = True

QUESTION 109
Which of the following scenarios is the best fit for the AUTO LOADER solution?

Efficiently process new data incrementally from cloud object storage

Incrementally process new streaming data from Apache Kafa into delta lake

Incrementally process new data from relational databases like MySQL

Efficiently copy data from data lake location to another data lake location

Efficiently move data incrementally from one delta table to another delta table

Explanation
The answer is, Efficiently process new data incrementally from cloud object storage.
Please note: AUTO LOADER only works on data/files located in cloud object storage like S3 or Azure Blob Storage it does not have the ability to read other data sources, although AU-TO LOADER is built on top of structured streaming it only supports files in the cloud object stor-age. If you want to use Apache Kafka then you can just use structured streaming.
Diagram Description automatically generated

Auto Loader and Cloud Storage Integration
Auto Loader supports a couple of ways to ingest data incrementally
1.Directory listing – List Directory and maintain the state in RocksDB, supports incremental file listing
2.File notification – Uses a trigger+queue to store the file notification which can be later used to retrieve the file, unlike Directory listing File notification can scale up to millions of files per day.
[OPTIONAL]
Auto Loader vs COPY INTO?
Auto Loader
Auto Loader incrementally and efficiently processes new data files as they arrive in cloud storage without any additional setup. Auto Loader provides a new Structured Streaming source called cloudFiles. Given an input directory path on the cloud file storage, the cloudFiles source automatically processes new files as they arrive, with the option of also processing existing files in that directory.
When to use Auto Loader instead of the COPY INTO?
*You want to load data from a file location that contains files in the order of millions or higher. Auto Loader can discover files more efficiently than the COPY INTO SQL command and can split file processing into multiple batches.
*You do not plan to load subsets of previously uploaded files. With Auto Loader, it can be more difficult to reprocess subsets of files. However, you can use the COPY INTO SQL command to reload subsets of files while an Auto Loader stream is simultaneously running.
Refer to more documentation here,
https://docs.microsoft.com/en-us/azure/databricks/ingestion/auto-loader

QUESTION 110
You are asked to write a python function that can read data from a delta table and return the Data-Frame, which of the following is correct?

Python function cannot return a DataFrame

Write SQL UDF to return a DataFrame

Write SQL UDF that can return tabular data

Python function will result in out of memory error due to data volume

Python function can return a DataFrame

QUESTION 111
Which of the below SQL commands create a Global temporary view?

1.CREATE OR REPLACE TEMPORARY VIEW view_name
2. AS SELECT * FROM table_name

1. CREATE OR REPLACE LOCAL TEMPORARY VIEW view_name
2. AS SELECT * FROM table_name

1. CREATE OR REPLACE GLOBAL TEMPORARY VIEW view_name
2. AS SELECT * FROM table_name
(Correct)

1.CREATE OR REPLACE VIEW view_name
2. AS SELECT * FROM table_name

1. CREATE OR REPLACE LOCAL VIEW view_name
2. AS SELECT * FROM table_name

QUESTION 112
Which of the following table constraints that can be enforced on Delta lake tables are supported?

Primary key, foreign key, Not Null, Check Constraints

Primary key, Not Null, Check Constraints

Default, Not Null, Check Constraints

Not Null, Check Constraints

Unique, Not Null, Check Constraints

QUESTION 113
You are using k-means clustering to classify heart patients for a hospital. You have chosen Patient Sex,
Height, Weight, Age and Income as measures and have used 3 clusters. When you create a pair-wise plot of
the clusters, you notice that there is significant overlap between the clusters. What should you do?

Identify additional measures to add to the analysis

Remove one of the measures

Decrease the number of clusters

Increase the number of clusters

QUESTION 114
A data engineer has set up two Jobs that each run nightly. The first Job starts at 12:00 AM, and it usually
completes in about 20 minutes. The second Job depends on the first Job, and it starts at 12:30 AM. Sometimes,
the second Job fails when the first Job does not complete by 12:30 AM.
Which of the following approaches can the data engineer use to avoid this problem?

They can set up a retry policy on the first Job to help it run more quickly

They can use cluster pools to help the Jobs run more efficiently

They can limit the size of the output in the second Job so that it will not fail as easily

They can set up the data to stream from the first Job to the second Job

They can utilize multiple tasks in a single job with a linear dependency

QUESTION 115
You are currently working with the second team and both teams are looking to modify the same notebook, you noticed that the second member is copying the notebooks to the personal folder to edit and replace the collaboration notebook, which notebook feature do you recommend to make the process easier to collaborate.

Databricks notebooks should be copied to a local machine and setup source control lo-cally to version the notebooks

Databricks notebooks support automatic change tracking and versioning

Databricks Notebooks support real-time coauthoring on a single notebook

Databricks notebooks can be exported into dbc archive files and stored in data lake

Databricks notebook can be exported as HTML and imported at a later time

QUESTION 116
When investigating a data issue you realized that a process accidentally updated the table, you want to query the same table with yesterday’s version of the data so you can review what the prior version looks like, what is the best way to query historical data so you can do your analysis?

SELECT * FROM TIME_TRAVEL(table_name) WHERE time_stamp = ‘timestamp’

TIME_TRAVEL FROM table_name WHERE time_stamp = date_sub(current_date(), 1)

SELECT * FROM table_name TIMESTAMP AS OF date_sub(current_date(), 1)

DISCRIBE HISTORY table_name AS OF date_sub(current_date(), 1)

SHOW HISTORY table_name AS OF date_sub(current_date(), 1)

QUESTION 117
You have accidentally deleted records from a table called transactions, what is the easiest way to restore the records deleted or the previous state of the table? Prior to deleting the version of the table is 3 and after delete the version of the table is 4.

RESTORE TABLE transactions FROM VERSION as of 4

RESTORE TABLE transactions TO VERSION as of 3
C .
1.INSERT INTO OVERWRITE transactions
2.SELECT * FROM transactions VERSION AS OF 3
3.MINUS
4.SELECT * FROM transactions

1.INSERT INTO OVERWRITE transactions
2.SELECT * FROM transactions VERSION AS OF 4
3.INTERSECT
4.SELECT * FROM transactions

COPY OVERWRITE transactions from VERSION as of 3

QUESTION 118
A dataset has been defined using Delta Live Tables and includes an expectations clause:
1. CONSTRAINT valid_timestamp EXPECT (timestamp > ‘2020-01-01’)
What is the expected behaviour when a batch of data containing data that violates these constraints is
processed?

Records that violate the expectation cause the job to fail

Records that violate the expectation are added to the target dataset and flagged as in-valid in a field added to the target dataset

Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table

Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log

Records that violate the expectation are added to the target dataset and recorded as invalid in the event log

QUESTION 119
How are Delt tables stored?

A Directory where parquet data files are stored, a sub directory _delta_log where meta data, and the transaction log is stored as JSON files.

A Directory where parquet data files are stored, all of the meta data is stored in memory

A Directory where parquet data files are stored in Data plane, a sub directory _delta_log where meta data, history and log is stored in control pane.

A Directory where parquet data files are stored, all of the metadata is stored in parquet files

Data is stored in Data plane and Metadata and delta log are stored in control pane

QUESTION 120
You are working on a process to query the table based on batch date, and batch date is an input parameter and expected to change every time the program runs, what is the best way to we can parameterize the query to run without manually changing the batch date?

Create a notebook parameter for batch date and assign the value to a python variable and use a spark data frame to filter the data based on the python variable

Create a dynamic view that can calculate the batch date automatically and use the view to query the data

There is no way we can combine python variable and spark code

Manually edit code every time to change the batch date

Store the batch date in the spark configuration and use a spark data frame to filter the data based on the spark configuration.

QUESTION 121
Which of the following results in the creation of an external table?

CREATE TABLE transactions (id int, desc string) USING DELTA LOCATION EX-TERNAL

CREATE TABLE transactions (id int, desc string)

CREATE EXTERNAL TABLE transactions (id int, desc string)

CREATE TABLE transactions (id int, desc string) TYPE EXTERNAL

CREATE TABLE transactions (id int, desc string) LOCATION ‘/mnt/delta/transactions’

QUESTION 122
In which phase of the data analytics lifecycle do Data Scientists spend the most time in a project?

Discovery

Data Preparation

Model Building

Communicate Results

QUESTION 123
While investigating a performance issue, you realized that you have too many small files for a given table, which command are you going to run to fix this issue

COMPACT table_name

VACUUM table_name

MERGE table_name

SHRINK table_name

OPTIMIZE table_name

QUESTION 124
You are trying to create an object by joining two tables that and it is accessible to data scientist’s team, so it does not get dropped if the cluster restarts or if the notebook is detached. What type of object are you trying to create?

Temporary view

Global Temporary view

Global Temporary view with cache option

External view

View

Databricks Certified Professional Data Engineer certification is designed for data engineers who are responsible for building and maintaining data pipelines and data lakes on the Databricks platform. Databricks Certified Professional Data Engineer Exam certification exam covers a wide range of topics, including data engineering concepts, data modeling, data ingestion, data transformation, data processing, and data warehousing. Databricks-Certified-Professional-Data-Engineer exam is designed to assess a candidate’s ability to design, build, and maintain scalable and reliable data pipelines on the Databricks platform.

Databricks Certified Professional Data Engineer certification is globally recognized by various industries, including finance, healthcare, government, and technology. Databricks Certified Professional Data Engineer Exam certification validates the candidate’s knowledge level in data engineering solutions and qualifies them to work with Databricks’s technology. The certified professional can optimize and manage their organization’s data in the cloud using Databricks, which results in timely and informed decisions.

Authentic Best resources for Databricks-Certified-Professional-Data-Engineer Online Practice Exam: https://www.braindumpstudy.com/Databricks-Certified-Professional-Data-Engineer_braindumps.html