Spark - {BigTechTalk}

April 3, 2024April 4, 2024scala, Spark

How to Access Relational Data with Apache Spark

Apache Spark, a powerful Apache framework for large-scale data processing, seamlessly integrates with relational databases using JDBC (Java Database Connectivity). This enables you to leverage Spark’s distributed processing capabilities to […]

March 31, 2024March 31, 2024scala, Spark

How to find table size in Apache Spark

In this article, we will explore techniques for determining the size of tables without scanning the entire dataset using the Spark Catalog API. Understanding table sizes is critical for optimizing […]

November 26, 2023March 28, 2024Spark

How to Read CSV file using Spark DataFrame

In this post i will try to explain how to read a csv file using spark and scala. Spark provides spark.read().csv(“path”) and spark.read.format(“csv”).load(“path”) In this tutorial, you will learn how […]

July 18, 2023April 5, 2024Spark

What is Session in Apache Spark

What is a Spark Session ? Spark Session was introduced in Spark 2.0 and it acts as an entry point into all functionality of Spark . Spark Session provides a […]

December 1, 2022March 19, 2024Spark

High Level Overview of Apache Spark

In this article, we are going to discuss the introductory part of Apache Spark, and the history of spark, and why spark is important. Let’s discuss one by one. According […]

November 27, 2022April 1, 2024Spark

How to Install Apache Spark on Windows 10

In this post, I’ll go through step-by-step instructions for installing Apache Spark on Windows 10 .If you already have Java 8 , you can skip the first steps. Install java […]

November 23, 2022March 19, 2024Spark

How to Install Spark on Ubuntu

In this article, we will cover the installation procedure of Apache Spark on the Ubuntu operating system. This article is divided into 4 parts. Install java on Ubuntu. Download the […]