As part of Azure role based certifications, DP-201 Designing an Azure Data Solution certification exam is the 2nd in the list to get the Azure Data Engineer Associate certificate. I had cleared DP-200 exam earlier. Yesterday, I cleared DP-201 and achieved the Associate certification.
This exam focusses on assessing the candidate in the following areas:
The primary skills are tested against the following core services from Azure
In the month of October, DigitalOcean in partnership with Intel and Dev are celebrating the Hacktoberfest event across the globe. This is an annual event which aims to increase awareness of open source in communities all over the world. There are meetups and events scheduled all over in this regards. If you are an open-source contributor, you can get a free t-shirt by registering for the event and submitting at least 4 pull requests during the month of October. You can find out more about the event from the website
The Hacktoberfest Singapore meetup was scheduled for Saturday 10 Octoberwith half-day event. I had the opportunity to present a topic on building DevOps pipelines with GitHub Actions. GitHub Actions allows us to automate workflows when certain events trigger on our repository. …
Following my virtual session to the Microsoft Cloud South Florida user group last week, I was invited to deliver a similar virtual event. This time it was for Dear Azure user group in India. I am thankful to Kasam Shaikh the organizer of this event. The event was scheduled for Sunday afternoon. Given the relaxed time of 12:30 PM IST, we decided to have a 2 hours session. This was the first time I was present to the vibrant community in India.
The theme of the session is similar to the last one. The main difference here was the duration. We scheduled it for 2 hours and that provided me with an opportunity to do a deep dive on some of the topics. We started off by looking at the components of the application consisting of a Web API as Producer for RabbitMQ. Then we looked at the consumer which is built as .Net Core executable. We went through the steps of building Docker images using Docker compose. …
In an earlier post / YouTube video, I had demonstrated how to automate code lining process using the GitHub Super Linter and GitHub action. In this post, we will explore how to use GitHub actions to automate the static code analysis using SonarCloud.
You might have heard about SonarQube. It offers scanners for different programming languages. SonarCloud is cloud service which scans codebases for bugs, vulnerabilities and code smells. At the point of this writing, there are 24 mainstream programming languages supported which include:
SonarCloud provides detailed analysis across multiple dimensions of the code. These are helpful in identifying common mistakes done by developers and ensure that the code is of high quality. SonarCloud will also give an indicator of how much time is required to fix all the reported issues and remove the technical debt. …
Due to the COVID-19 pandemic, many of the developer communities and user groups have been forced to conduct their regular session in a virtual manner. This has provided a great opportunity for organizers and speakers from across the globe to speak at community events and rope in speakers from different parts of the world. This might not have been possible in case of physical events.
I have been speaking at the local community events in Singapore as well as other parts of Asia for the past 3–4 years. …
As developers, we work on multiple programming languages like C#, Java, Python, JavaScript, SQL, etc. Apart from the mainstream programming languages, we also work with different file types like XML, JSON, YAML. Each of these languages and file types has their own styles and conventions. As a language or a format becomes mature, there are standards and best practices which get developed over a period of time.
Nowadays, many developers are full stack developers or polyglot developers. As part of the Microservices style of development, they might work on the web or Javascript based front end, a Java, C#, Go, Python or some other programming language based middle tier. And then some SQL or NoSQL backend. Each of these components can use a different language. And each language will have its own style. …
Most of us are working from home due to the ongoing COVID-19 situation. This has saved some time in travelling for me personally. I managed to utilize some of this time to prepare for Azure Role-based certifications. In the past 3 months, I managed to clear AZ-301, AZ-400 and DP-200 certification exams. Many people have asked me about how did I go about preparing for these certifications. This post and the accompanying YouTube video is about my preparation for the DP-200 exam.
In case you are new to Microsoft Role-based certifications, have a look at this excellent post by Thomas Maurer about selecting the right exam. The DP 200 exam is focused on Data Engineer role and falls under the Data & AI category of tests. It is for the Associate level. I have been part of Data Engineering teams since late 2015. I have worked with Big Data technologies on Hortonworks Data Platform and also on Microsoft Azure. …
There are multiple ways of installing software packages. These can include common utilities such as 7-zip or Winzip, browsers such as Google Chrome, Firefox, Brave etc. We also install editors like Notepad Plus Plus, Visual Studio Code, terminals such as Fluent Terminal, cmder etc. The list can just go on and on. Things get complicated when you move from one operating system to another like MacOS or Linux. Lets add one more twist by adding the processor architecture or type 32 bit or 64 bit, Intel or AMD processors.
The way we install software could vary based on many of the factors listed above. Even for technical people, it can become challenge sometimes to identify the right version of the software to download and install. …
There are different ways in which we can manage environment specific settings related to containerized applications. Kubernetes provides ConfigMaps and Secrets as two options to manage environment configuration. ConfigMaps are good for storing key value pairs in plain text. When we are dealing with sensitive information containing connection strings, user name and password, certificates etc. these should be stored in encrypted form. Kubernetes Secret objects stores data in Base64 encoded form.
Storing secrets in encrypted form provides first line of defense. As the popularity of Kubernetes increases, the tools surrounding the ecosystem are also improving on regular basis. …
When we start working with Docker and Kubernetes, we need a container registry to publish out images. Most people start with the public DockerHub registry. This is good for getting started. however, as we become more proficient with container images and orchestration using something like Kubernetes, we need enterprise grade features. The public container registries do not provide all the enterprise grade features. In such scenario, we need to choose private container registry. In this post we will see how to integrate private Azure Container Registry (ACR) with Azure Kubernetes Service (AKS) cluster.
Similar to other private container registries, ACR provides following…
About