I have experience in the complete software lifecycle. To find out more details, consult my case study and online resume.
C#/.NET
C# 11 core language, Tasks, iterators, LINQ and Parallel LINQ, memory and span, array pools, functional programming, static abstract interface methods, abstract maths, libraries: GRPC, JSON, Kafka.
Scala
Object oriented and functional paradigms, trait composition, generic structures, collections, functional operators, Java interop, Sbt, Kafka, unit testing, XML library.
Apache Spark
Spark SQL, RDDs, data sets, JDBC, CSV, Kafka, HDFS, broadcasting, Delta Lake, Hive, logging and monitoring.
Azure
Blob storage, Azure Functions, HD Insights, Synapse, Monitor Logs, Kusto (KQL), Azure CLI, Application Insights, Data Factory, Databricks, Pipelines, DevOps.
PowerShell
Scripting, configuration, automation, file management, log filtering, XML, JSON, HTML and REST inspection, remote operations, ad-hoc testing.
Software Design
Design for maintainability, modularity, programming paradigms (object oriented, functional, procedural), programming best practices.
Correctness
Design patterns, unit and integration testing, debugging, profiling, diagnostics.
Software Engineering
Versioning, deployment, dependency analysis, automation, scripting, command line tools, Docker, build systems.
Big Data
Data pipelines, data cleaning, ingestion, transformation, Azure Data Factory, ETL, JSON and CSV, data warehouse (Synapse, MySql).
Case Studies
Big Data Pipeline (2022/2023)
In this article, a Big Data project I worked in from 2022 through 2023 is reviewed.