Module 1: What is Data PipelineWhat is Azure databricksAzure Databricks ArchitectureAzure Account SetupWorkSpace SetupModule 2:Navigate the WorkspaceRuntimesClusters NotebooksLibrariesRepos Databricks File System (DBFS)DBUTILSWidgetsWorkflowsMetastore - Setup external MetastoreModule 3: What is RDD Creating RDDRDD transformationsRDD ActionsRDD JoinsPair RDDBroadcast VariablesAccumulatorsConvert RDD to DataFrameImport & Read dataCreate a table using the UICreate a table in a NotebookModule 4: Create DataFramesDefine SchemaFunctionsCasting OperationsFilter TransformationUpdate, Update ALL & UpdateByNameOrderBy & SortBYGroupByRemove DuplicatesWindow FunctionsDate and Timestamp FunctionsUDF (User Defined Function)JOINHandle corrupt records using the badRecordsPathFile metadata columnModule 5: Read Parquet FileRead CSV FilesRead JSON FilesRead XML FilesRead Excel fileSQL databases using JDBCAzure blob storageModule 6: What is Spark Structure StreamingData Source & SinkRate & File SourceKafka SourceSink: Console, Memory, File & CustomBuild Streaming ETLStream ETL 1: Setup Event HubStreaming ETL 2: Event Hub ProducerStreaming ETL 3: Integrate Event Hubs with Data BricksStreaming ETL 4: TransformationStreaming ETL 5: Ingest into Azure Data storageTwitter Sentiment Analysis - IntroductionSetup Twitter Developer AccountTwitter Sentiment Analysis - IITwitter Sentiment Analysis - IIIModule 7: Components in Databricks SQLConfiguring a SQL EndpointCreating a Table from a CSV FileCreate QueriesParameterized QueryQuery ProfileBuilding Visualization (Table, BAR & PIE )Building Line Chart & Counter ChartAdding Charts to DashboardsDefining a Query AlertAccess Control on Databricks SQL ObjectsLab: Data Object Access ControlTransfer OwnershipAccess SQL Endpoint from PythonDatabricks SQL CLIDatabricks SQL CLI
