Latest Success Metrics For Actual Databricks-Certified-Professional-Data-Engineer Exam 2023 Realistic Dumps
Updated Databricks-Certified-Professional-Data-Engineer Dumps Questions For Databricks Exam
NEW QUESTION # 16
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE.
Three datasets are defined against Delta Lake table sources using LIVE TABLE . The table is configured to
run in Development mode using the Triggered Pipeline Mode.
Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after
clicking Start to update the pipeline?
- A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will
persist after the pipeline is stopped to allow for additional testing - B. All datasets will be updated continuously and the pipeline will not shut down. The compute resources
will persist with the pipeline - C. All datasets will be updated once and the pipeline will shut down. The compute resources will be
terminated - D. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to
allow for additional testing - E. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will
be deployed for the update and terminated when the pipeline is stopped
Answer: D
NEW QUESTION # 17
A SQL Dashboard was built for the supply chain team to monitor the inventory and product orders, but all of the timestamps displayed on the dashboards are showing in UTC format, so they requested to change the time zone to the location of New York. How would you approach resolving this issue?
- A. Under SQL Admin Console, set the SQL configuration parameter time zone to Ameri-ca/New_York
- B. Add SET Timezone = America/New_York on every of the SQL queries in the dashboard.
- C. Change the spark configuration of SQL endpoint to format the timestamp to Ameri-ca/New_York
- D. Move the workspace from Central US zone to East US Zone
- E. Change the timestamp on the delta tables to America/New_York format
Answer: A
Explanation:
Explanation
The answer is, Under SQL Admin Console, set the SQL configuration parameter time zone to America/New_York Here are steps you can take this to configure, so the entire dashboard is changed without changing individual queries Configure SQL parameters To configure all warehouses with SQL parameters:
1.Click Settings at the bottom of the sidebar and select SQL Admin Console.
2.Click the SQL Warehouse Settings tab.
3.In the SQL Configuration Parameters textbox, specify one key-value pair per line. Sepa-rate the name of the parameter from its value using a space. For example, to ena-ble ANSI_MODE:
Graphical user interface, text, application Description automatically generated
Similarly, we can add a line in the SQL Configuration parameters
timezone America/New_York
SQL configuration parameters | Databricks on AWS
NEW QUESTION # 18
You are working on a marketing team request to identify customers with the same information between two tables CUSTOMERS_2021 and CUSTOMERS_2020 each table contains 25 columns with the same schema, You are looking to identify rows that match between two tables across all columns, which of the following can be used to perform in SQL
- A. 1.SELECT * FROM CUSTOMERS_2021
2.EXCEPT
3.SELECT * FROM CUSTOMERS_2020 - B. 1.SELECT * FROM CUSTOMERS_2021
2. UNION
3.SELECT * FROM CUSTOMERS_2020 - C. 1.SELECT * FROM CUSTOMERS_2021 C1
2.INNER JOIN CUSTOMERS_2020 C2
3.ON C1.CUSTOMER_ID = C2.CUSTOMER_ID - D. 1.SELECT * FROM CUSTOMERS_2021
2. UNION ALL
3.SELECT * FROM CUSTOMERS_2020 - E. 1.SELECT * FROM CUSTOMERS_2021
2. INTERSECT
3.SELECT * FROM CUSTOMERS_2020
Answer: E
Explanation:
Explanation
Answer is,
1.SELECT * FROM CUSTOMERS_2021
2. INTERSECT
3.SELECT * FROM CUSTOMERS_2020
To compare all the rows between both the tables across all the columns using intersect will help us achieve that, an inner join is only going to check if the same column value exists across both the tables on a single column.
INTERSECT [ALL | DISTINCT]
*Returns the set of rows which are in both subqueries.
If ALL is specified a row that appears multiple times in the subquery1 as well as in subquery will be returned multiple times.
If DISTINCT is specified the result does not contain duplicate rows. This is the default.
NEW QUESTION # 19
Which of the following are stored in the control pane of Databricks Architecture?
- A. Databricks Web Application
- B. All Purpose Clusters
- C. Job Clusters
- D. Delta tables
- E. Databricks Filesystem
Answer: A
Explanation:
Explanation
The answer is Databricks Web Application
Azure Databricks architecture overview - Azure Databricks | Microsoft Docs Databricks operates most of its services out of a control plane and a data plane, please note serverless features like SQL Endpoint and DLT compute use shared compute in Control pane.
Control Plane: Stored in Databricks Cloud Account
* The control plane includes the backend services that Databricks manages in its own Azure account.
Notebook commands and many other workspace configurations are stored in the control plane and encrypted at rest.
Data Plane: Stored in Customer Cloud Account
* The data plane is managed by your Azure account and is where your data resides. This is also where data is processed. You can use Azure Databricks connectors so that your clusters can connect to external data sources outside of your Azure account to ingest data or for storage.
Timeline Description automatically generated
Bottom of Form
Top of Form
NEW QUESTION # 20
What is the underlying technology that makes the Auto Loader work?
- A. Structured Streaming
- B. Delta Live Tables
- C. DataFrames
- D. Live DataFames
- E. Loader
Answer: A
NEW QUESTION # 21
What is the top-level object in unity catalog?
- A. Table
- B. Catalog
- C. Database
- D. Workspace
- E. Metastore
Answer: E
Explanation:
Explanation
Key concepts - Azure Databricks | Microsoft Docs
Diagram Description automatically generated
NEW QUESTION # 22
Which of the following commands results in the successful creation of a view on top of the delta stream(stream on delta table)?
- A. Spark.read.format("delta").table("sales").mode("stream").createOrReplaceTempView("streaming_vw")
- B. You can not create a view on streaming data source.
- C. Spark.read.format("delta").stream("sales").createOrReplaceTempView("streaming_vw")
- D. Spark.read.format("delta").table("sales").createOrReplaceTempView("streaming_vw")
- E. Spark.readStream.format("delta").table("sales").createOrReplaceTempView("streaming_vw")
- F. Spark.read.format("delta").table("sales").trigger("stream").createOrReplaceTempView("streaming_vw")
Answer: E
Explanation:
Explanation
The answer is
Spark.readStream.table("sales").createOrReplaceTempView("streaming_vw") When you load a Delta table as a stream source and use it in a streaming query, the query processes all of the data present in the table as well as any new data that arrives after the stream is started.
You can load both paths and tables as a stream, you also have the ability to ignore deletes and changes(updates, Merge, overwrites) on the delta table.
Here is more information,
https://docs.databricks.com/delta/delta-streaming.html#delta-table-as-a-source
NEW QUESTION # 23
Which of the following SQL statements can be used to update a transactions table, to set a flag on the table from Y to N
- A. MODIFY transactions SET active_flag = 'N' WHERE active_flag = 'Y'
- B. MERGE transactions SET active_flag = 'N' WHERE active_flag = 'Y'
- C. REPLACE transactions SET active_flag = 'N' WHERE active_flag = 'Y'
- D. UPDATE transactions SET active_flag = 'N' WHERE active_flag = 'Y'
Answer: C
Explanation:
Explanation
The answer is
UPDATE transactions SET active_flag = 'N' WHERE active_flag = 'Y'
Delta Lake supports UPDATE statements on the delta table, all of the changes as part of the update are ACID compliant.
NEW QUESTION # 24
An upstream source writes Parquet data as hourly batches to directories named with the current date. A nightly batch job runs the following code to ingest all data from the previous day as indicated by thedatevariable:
Assume that the fieldscustomer_idandorder_idserve as a composite key to uniquely identify each order.
If the upstream system is known to occasionally produce duplicate entries for a single order hours apart, which statement is correct?
- A. Each write to the orders table willonly contain unique records, and only those records without duplicates in the target table will be written.
- B. Each write to the orders table willrun deduplication over the union of new and existing records, ensuringno duplicate records are present.
- C. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, the operation will tail.
- D. Each write to the orders table will only contain unique records; if existing records with the same key are present in the target table, these records will be overwritten.
- E. Each write to the orders table willonly contain unique records, but newly written records may have duplicates already present in the target table.
Answer: E
Explanation:
Explanation
This is the correct answer because the code uses the dropDuplicates method to remove any duplicate records within each batch of data before writing to the orders table. However, this method does not check for duplicates across different batches or in the target table, so it is possible that newly written records may have duplicates already present in the target table. To avoid this, a better approach would be to use Delta Lake and perform an upsert operation usingmergeInto. Verified References: [Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "DROP DUPLICATES" section.
NEW QUESTION # 25
Which of the following is correct for the global temporary view?
- A. global temporary views cannot be accessed once the notebook is detached and attached
- B. global temporary views can be still accessed even if the cluster is restarted
- C. global temporary views are created in a database called temp database
- D. global temporary views can be accessed across many clusters
- E. global temporary views can be still accessed even if the notebook is detached and at-tached
Answer: E
Explanation:
Explanation
The answer is global temporary views can be still accessed even if the notebook is detached and attached There are two types of temporary views that can be created Local and Global
* A local temporary view is only available with a spark session, so another notebook in the same cluster can not access it. if a notebook is detached and reattached local temporary view is lost.
* A global temporary view is available to all the notebooks in the cluster, even if the notebook is detached and reattached it can still be accessible but if a cluster is restarted the global temporary view is lost.
NEW QUESTION # 26
A data engineer is using a Databricks SQL query to monitor the performance of an ELT job. The ELT job is triggered by a specific number of input records being ready to process. The Databricks SQL query returns the number of minutes since the job's most recent runtime. Which of the following approaches can enable the data engineering team to be notified if the ELT job has not been run in an hour?
- A. They can set up an Alert for the query to notify them if the returned value is greater than 60.
- B. They can set up an Alert for the query to notify when the ELT job fails.
- C. They can set up an Alert for the accompanying dashboard to notify them if the returned value is greater than 60.
- D. This type of alert is not possible in Databricks
- E. They can set up an Alert for the accompanying dashboard to notify when it has not re-freshed in 60 minutes.
Answer: A
Explanation:
Explanation
The answer is, They can set up an Alert for the query to notify them if the returned value is greater than 60.
The important thing to note here is that alert can only be setup on query not on the dashboard, query can return a value, which is used if alert can be triggered.
NEW QUESTION # 27
Although the Databricks Utilities Secrets module provides tools to store sensitive credentials and avoid accidentally displaying them in plain text users should still be careful with which credentials are stored here and which users have access to using these secrets.
Which statement describes a limitation of Databricks Secrets?
- A. Secrets are stored in an administrators-only table within the Hive Metastore; database administrators have permission to query this table by default.
- B. The Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials.
- C. Account administrators can see all secrets in plain text by loggingon to the Databricks Accounts console.
- D. Because the SHA256 hash is used to obfuscate stored secrets, reversing this hash will display the value in plain text.
- E. Iterating through a stored secret and printing each character will display secret contents in plain text.
Answer: B
Explanation:
Explanation
This is the correct answer because it describes a limitation of Databricks Secrets. Databricks Secrets is a module that provides tools to store sensitive credentials and avoid accidentally displaying them in plain text.
Databricks Secrets allows creating secret scopes, which are collections of secrets that can be accessed by users or groups. Databricks Secrets also allows creating and managing secrets using the Databricks CLI or the Databricks REST API. However, a limitation of Databricks Secrets is that the Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials. Therefore, users should still be careful with which credentials are stored in Databricks Secrets and which users have access to using these secrets. Verified References: [Databricks Certified Data Engineer Professional], under "Databricks Workspace" section; Databricks Documentation, under "List secrets" section.
NEW QUESTION # 28
You are working to set up two notebooks to run on a schedule, the second notebook is dependent on the first notebook but both notebooks need different types of compute to run in an optimal fashion, what is the best way to set up these notebooks as jobs?
- A. Use a very large cluster to run both the tasks in a single job
- B. Use DELTA LIVE PIPELINES instead of notebook tasks
- C. Each task can use different cluster, add these two notebooks as two tasks in a single job with linear dependency and modify the cluster as needed for each of the tasks
- D. Use a single job to setup both notebooks as individual tasks, but use the cluster API to setup the second cluster before the start of second task
- E. A Job can only use single cluster, setup job for each notebook and use job dependency to link both jobs together
Answer: C
Explanation:
Explanation
Tasks in Jobs support different clusters for each task in the same job.
NEW QUESTION # 29
A small company based in the United States has recently contracted a consulting firm in India to implement several new data engineering pipelines to power artificial intelligence applications. All the company's data is stored in regional cloud storage in the United States.
The workspace administrator at the company is uncertain about where the Databricks workspace used by the contractors should be deployed.
Assuming that all data governance considerations are accounted for, which statement accurately informs this decision?
- A. Databricks workspaces do not rely on any regional infrastructure; as such, the decision should be made based upon what is most convenient for the workspace administrator.
- B. Databricks runs HDFS on cloud volume storage; as such, cloud virtual machines must be deployed in the region where the data is stored.
- C. Databricks notebooks send all executable code from the user's browser to virtual machines over the open internet; whenever possible, choosing a workspace region near the end users is the most secure.
- D. Databricks leverages user workstations as the driver during interactive development; as such, users should always use a workspace deployed in a region they are physically near.
- E. Cross-region reads and writes can incur significant costs and latency; whenever possible, compute should be deployed in the same region the data is stored.
Answer: E
Explanation:
Explanation
This is the correct answer because it accurately informs this decision. The decision is about where the Databricks workspace used by the contractors should be deployed. The contractors are based in India, while all the company's data is stored in regional cloud storage in the United States. When choosing a region for deploying a Databricks workspace, one of the important factors to consider is the proximity to the data sources and sinks. Cross-region reads and writes can incur significant costs and latency due to network bandwidth and data transfer fees. Therefore, whenever possible, compute should be deployed in the same region the data is stored to optimize performance and reduce costs. Verified References: [Databricks Certified Data Engineer Professional], under "Databricks Workspace" section; Databricks Documentation, under "Choose a region" section.
NEW QUESTION # 30
Which of the following Structured Streaming queries is performing a hop from a Bronze table to a Silver
table?
- A. 1. (spark.table("sales")
2. .withColumn("avgPrice", col("sales") / col("units"))
3. .writeStream
4. .option("checkpointLocation", checkpointPath)
5. .outputMode("append")
6. .table("cleanedSales")
7.) - B. 1. (spark.readStream.load(rawSalesLocation)
2. .writeStream
3. .option("checkpointLocation", checkpointPath)
4. .outputMode("append")
5. .table("uncleanedSales")
6. ) - C. 1. (spark.table("sales")
2. .groupBy("store")
3. .agg(sum("sales"))
4. .writeStream
5. .option("checkpointLocation", checkpointPath)
6. .outputMode("complete")
7. .table("aggregatedSales")
8.) - D. 1. (spark.read.load(rawSalesLocation)
2. .writeStream
3. .option("checkpointLocation", checkpointPath)
4. .outputMode("append")
5. .table("uncleanedSales")
6. ) - E. 1. (spark.table("sales")
2. .agg(sum("sales"),
3. sum("units"))
4. .writeStream
5. .option("checkpointLocation", checkpointPath)
6. .outputMode("complete")
7. .table("aggregatedSales")
8. )
Answer: A
NEW QUESTION # 31
Which of the following commands can be used to run one notebook from another notebook?
- A. dbutils.notebook.run("full notebook path")
- B. only job clusters can run notebook
- C. execute.utils.run("full notebook path")
- D. spark.notebook.run("full notebook path")
- E. notebook.utils.run("full notebook path")
Answer: A
Explanation:
Explanation
The answer is dbutils.notebook.run(" full notebook path ")
Here is the full command with additional options.
run(path: String, timeout_seconds: int, arguments: Map): String
1.dbutils.notebook.run("ful-notebook-name", 60, {"argument": "data", "argument2": "data2", ...})
NEW QUESTION # 32
How do you access or use tables in the unity catalog?
- A. schema_name.catalog_name.table_name
- B. schema_name.table_name
- C. catalog_name.schema_name.table_name
- D. catalog_name.table_name
- E. catalog_name.database_name.schema_name.table_name
Answer: C
Explanation:
Explanation
The answer is catalog_name.schema_name.table_name
Graphical user interface, diagram Description automatically generated
Note: Database and Schema are analogous they are interchangeably used in the Unity catalog.
FYI, A catalog is registered under a metastore, by default every workspace has a default metastore called hive_metastore, with a unity catalog you have the ability to create meatstores and share that across multiple workspaces.
Diagram Description automatically generated
NEW QUESTION # 33
Which one of the following is not a Databricks lakehouse object?
- A. Functions
- B. Database/Schemas
- C. Views
- D. Stored Procedures
- E. Catalog
- F. Tables
Answer: D
Explanation:
Explanation
The answer is, Stored Procedures.
Databricks lakehouse does not support stored procedures.
NEW QUESTION # 34
A data engineer has a Job with multiple tasks that runs nightly. One of the tasks unexpectedly fails during 10
percent of the runs.
Which of the following actions can the data engineer perform to ensure the Job completes each night while
minimizing compute costs?
- A. They can set up the Job to run multiple times ensuring that at least one will complete
- B. They can utilize a Jobs cluster for each of the tasks in the Job
- C. They can observe the task as it runs to try and determine why it is failing
- D. They can institute a retry policy for the task that periodically fails
- E. They can institute a retry policy for the entire Job
Answer: D
NEW QUESTION # 35
What could be the expected output of query SELECT COUNT (DISTINCT *) FROM user on this table
- A. 0
- B. 1
- C. NULL
- D. 2
(Correct) - E. 2
Answer: D
Explanation:
Explanation
The answer is 2,
Count(DISTINCT *) removes rows with any column with a NULL value
NEW QUESTION # 36
Which of the following SQL statements can replace a python variable, when the notebook is set in SQL mode
1.table_name = "sales"
2.schema_name = "bronze"
- A. spark.sql(f"SELECT * FROM {schema_name}.{table_name}")
- B. spark.sql(f"SELECT * FROM {schem_name.table_name}")
- C. spark.sql(f"SELECT * FROM ${schema_name}.${table_name}")
- D. spark.sql("SELECT * FROM schema_name.table_name")
- E. spark.sql(f"SELECT * FROM f{schema_name.table_name}")
Answer: A
Explanation:
Explanation
The answer is spark.sql(f"SELECT * FROM {schema_name}.{table_name}")
NEW QUESTION # 37
You are currently working on a notebook that will populate a reporting table for downstream process consumption, this process needs to run on a schedule every hour, what type of cluster are you going to use to set up this job?
- A. The job cluster is best suited for this purpose.
- B. Use delta live table pipeline to run in continuous mode
- C. Since it's just a single job and we need to run every hour, we can use an all-purpose cluster
- D. Use Azure VM to read and write delta tables in Python
Answer: A
Explanation:
Explanation
The answer is, The Job cluster is best suited for this purpose.
Since you don't need to interact with the notebook during the execution especially when it's a scheduled job, job cluster makes sense. Using an all-purpose cluster can be twice as expensive as a job cluster.
FYI,
When you run a job scheduler with option of creating a new cluster when the job is complete it terminates the cluster. You cannot restart a job cluster.
NEW QUESTION # 38
A newly joined team member John Smith in the Marketing team currently has access read access to sales tables but does not have access to update the table, which of the following commands help you accomplish this?
- A. GRANT UPDATE TO TABLE table_name ON [email protected]
- B. GRANT USAGE ON TABLE table_name TO [email protected]
- C. GRANT MODIFY ON TABLE table_name TO [email protected]
- D. GRANT UPDATE ON TABLE table_name TO [email protected]
- E. GRANT MODIFY TO TABLE table_name ON [email protected]
Answer: C
Explanation:
Explanation
The answer is GRANT MODIFY ON TABLE table_name TO [email protected]
https://docs.microsoft.com/en-us/azure/databricks/security/access-control/table-acls/object-privileges#privileges
NEW QUESTION # 39
......
Full Databricks-Certified-Professional-Data-Engineer Practice Test and 60 Unique Questions, Get it Now!: https://www.itcertmagic.com/Databricks/real-Databricks-Certified-Professional-Data-Engineer-exam-prep-dumps.html