If you're reading this, you're probably stuck in the same nightmare I was: Antigravity shows "Authentication Required Please sign in" no matter what you do. You've reinstalled it. You've cleared the browser cache. You've restarted your MacBook. Nothing works.

Here's the complete story of what happened, every fix I tried, and what finally resolved it.

The symptoms were confusing and contradictory:

• Antigravity's Settings page showed my Google account, so it knew I was logged in

• But the main/Agent view kept showing "Sign In"

• Clicking "Sign In" would open Google auth in the browser, I'd complete it successfully, but Antigravity would loop right back to "Authentication Required"

• I couldn't even sign out, there was no sign-out option because the app was in a broken half-authenticated state

What I Already Tried (That Didn't Work)

Before finding the real fix, I tried everything the internet suggested:

• ❌ Restarted the MacBook

• ❌ Cleared browser cache and cookies

• ❌ Deleted Antigravity and reinstalled it

• ❌ Deleted ~/Library/Application Support/Antigravity/Session Storage

• ❌ Deleted ~/Library/Application Support/Antigravity/Local Storage

• ❌ Deleted auth-tokens, Cookies folders

• ❌ Deleted ~/.gemini/antigravity

• ❌ Used Ctrl+Shift+P → Antigravity: Reset Onboarding

• ❌ Applied the BigInt patch to main.js

• ❌ Switched default browser from Brave to Safari

None of it worked. Here's why, I was missing the actual root cause every single time.

The Root Cause: macOS Keychain

Here's what nobody tells you:

When Antigravity first authenticates, it stores an encrypted token in macOS Keychain under the name "Antigravity Safe Storage". This is completely separate from the app's files in ~/Library/Application Support/.

When you uninstall Antigravity, the app files are removed, but the Keychain entry stays behind. When you reinstall and try to log in again, Antigravity finds this old, expired, corrupted token in Keychain and tries to use it. It fails silently. No error message. Just "Authentication Required" forever.

This is why:

• Other macOS user profiles on the same Mac worked fine (each profile has its own Keychain)

• Reinstalling never helped (Keychain survives uninstalls)

• Clearing Library folders never helped (Keychain is stored separately)

The Fix

Step 1: Kill Antigravity completely

pkill -f Antigravity

Step 2: Delete all app data

rm -rf ~/Library/Application\ Support/Antigravity/
rm -rf ~/.gemini/antigravity
rm -rf ~/Library/Caches/Antigravity/
rm -rf ~/Library/Logs/Antigravity/

Step 3: Delete the Keychain entry (THE KEY STEP)

security delete-generic-password -l "Antigravity Safe Storage"

You should see:

password has been deleted.

Alternatively, do it via UI:

1. Open Keychain Access (Spotlight → search "Keychain Access")

2. Search for antigravity

3. Right-click "Antigravity Safe Storage" → Delete

4. Enter your Mac login password when prompted → click Always Allow

Step 4: Set Safari as default browser temporarily

• Apple menu → System Settings → Desktop & Dock → Default web browser → Safari

  Why Safari? If you use Brave, its aggressive "Auto-redirect tracking URLs" feature intercepts the OAuth callback URL and silently kills the token handoff back to the app.

Step 5: Sign in

1. Open Antigravity

2. Click Sign In

3. Complete Google auth in Safari

4. ✅ Done, switch back to Brave after

Bonus: Why Brave Browser Made It Worse

Even after fixing the Keychain issue, if you use Brave as your default browser, you may still face the auth loop. Brave's Shields intercept OAuth redirect URLs as "tracking URLs" and kill the token handoff. The fix is simple, use Safari (or any non-Brave browser) as your default browser just for the sign-in step.

Conclusion

The corrupted Keychain entry in my case was dated November 21, 2025. A 4-month-old stale token that had been silently blocking every login attempt since then.

If you're stuck on this issue, go straight to the Keychain step. It'll save you hours.

Every Kubernetes distribution needs to prove it behaves like Kubernetes. That proof comes through the CNCF Kubernetes Conformance Program, a standardized set of 424 end-to-end tests that verify a distribution implements the Kubernetes API correctly. If it passes, it earns the "Certified Kubernetes" badge.

This is the story of how I picked up a GitHub issue, ran into DNS nightmares, broke my cluster twice, learned a ton about Linux internals, and ultimately got my PR merged into the official CNCF conformance repository.

What is Kubespray?

Kubespray is an Ansible-based tool for deploying production-ready Kubernetes clusters. It's part of the official kubernetes-sigs organization and supports deployment across AWS, GCE, Azure, OpenStack, vSphere, Equinix Metal (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal

Think of it as "Infrastructure as Code for Kubernetes", you define your cluster configuration, and Kubespray handles the 50+ steps needed to get a production-grade Kubernetes cluster running. For a deeper dive, check the official documentation.

What is Conformance Testing?

The CNCF runs the Kubernetes Software Conformance Certification program. Every time a new Kubernetes version is released, distributions like Kubespray need to submit proof that they can deploy a conformant cluster.

The submission requires four files:

The testing tool is Sonobuoy, it runs 424 end-to-end tests inside your cluster in certified-conformance mode.

The Issue: K8s Conformance Test for v1.34

I found issue #12932 on the kubespray repo, they needed someone to run the conformance tests for Kubernetes v1.34 using Kubespray v2.30.0 and submit the results to the CNCF.

The task seemed straightforward:

1. Spin up a 3-node Kubernetes cluster using Kubespray

2. Run Sonobuoy conformance tests

3. Collect results and submit a PR

How hard could it be? Spoiler: very.

My Setup

Why Vagrant?

Vagrant is a tool for building and managing virtual machine environments. It reads a Vagrantfile and provisions VMs automatically. Kubespray includes a Vagrantfile that integrates with its Ansible playbooks, so vagrant up creates VMs, configures networking, and runs Kubespray to deploy Kubernetes. All in one command.

For conformance testing, this is ideal because it ensures reproducibility, anyone can follow the same steps to recreate the exact environment.

Why Calico Instead of Flannel?

This is an ARM64-specific gotcha. According to the Kubespray architecture docs, Flannel does not support ARM64. Calico is the recommended CNI for ARM64 clusters.

The Journey: What Actually Happened

Phase 1: Getting the Cluster Up

Setting up the Vagrant configuration was straightforward:

$instance_name_prefix = "kube"
$vm_cpus = 2
$vm_memory = 4096
$num_instances = 3
$os = "ubuntu2404"
$subnet = "10.2.20"
$inventory = "inventory/k8s-conformance"
$network_plugin = "calico"

Running vagrant up kicked off the Ansible playbook, and after about 20 minutes, I had a 3-node Kubernetes cluster. So far, so good.

$ kubectl get nodes
NAME     STATUS   ROLES           AGE   VERSION
kube-1   Ready    control-plane   10m   v1.34.3
kube-2   Ready    control-plane   10m   v1.34.3
kube-3   Ready    <none>          10m   v1.34.3

Phase 2: The First Sonobuoy Run — DNS Failures

I downloaded Sonobuoy v0.57.2 (ARM64 binary) and started the conformance tests:

./sonobuoy run --mode=certified-conformance \
    --sonobuoy-image=sonobuoy/sonobuoy:v0.57 \
    --systemd-logs-image=sonobuoy/systemd-logs-arm64:v0.4 \
    --wait

~2 hours later, the results came back:

Passed: 0, Failed: 0, Remaining: 424

Wait, 0 passed AND 0 failed? That means the e2e plugin crashed before running any tests. Digging into the logs, I found the culprit:

[sig-network] DNS should provide DNS for the cluster
FAILED [602.517 seconds]

Unable to read from pod dns-test-...: the server could not find the requested resource

The test pods were being created but immediately disappearing. The DNS test couldn't find its own pods.

Phase 3: The Rabbit Hole — systemd-resolved

Here's where it got interesting. Ubuntu 24.04 uses systemd-resolved for DNS resolution. This service creates a stub resolver at 127.0.0.53 and manages /etc/resolv.conf(file:///etc/resolv.conf) as a symlink to /run/systemd/resolve/stub-resolv.conf.

The problem: CoreDNS inside the cluster was trying to use 127.0.0.53 as an upstream DNS server, but that address is only valid inside the host, not inside containers. Every container DNS lookup was timing out.

The fix:

# Disable systemd-resolved
sudo systemctl stop systemd-resolved
sudo systemctl disable systemd-resolved

# Replace with direct DNS servers
sudo rm -f /etc/resolv.conf
echo -e 'nameserver 8.8.8.8\nnameserver 8.8.4.4' | sudo tee /etc/resolv.conf

I applied this on all three nodes. DNS test looked good now:

$ kubectl run dnstest --image=busybox:1.28 --rm -it --restart=Never -- nslookup google.com
Name:      google.com
Address 1: 142.250.192.206
✅

Phase 4: The Second Sonobuoy Run — Still Failing

Ran Sonobuoy again. Another 2+ hours. Same result:

Passed: 0, Failed: 0, Remaining: 424

The cluster was showing signs of instability, controller restarts, unhealthy probes, context deadline exceeded errors. Something deeper was wrong.

Phase 5: The Reboot That Broke Everything

I rebooted all three nodes to clear the instability. After rebooting, the cluster was completely dead:

The connection to the server 127.0.0.1:6443 was refused

The API server wasn't starting. No containers were running. Checking the kubelet logs revealed the smoking gun:

E0203 18:15:08.841453 dns.go:285] "Could not open resolv conf file."
err="open /run/systemd/resolve/resolv.conf: no such file or directory"

E0203 18:15:08.841474 kuberuntime_sandbox.go:44]
"Failed to generate sandbox config for pod"
err="open /run/systemd/resolve/resolv.conf: no such file or directory"

Our DNS fix broke the kubelet!

When we disabled systemd-resolved and deleted its files, we also removed /run/systemd/resolve/resolv.conf. But the kubelet was configured to read DNS settings from that exact file path. Without it, kubelet couldn't create any pod sandboxes — which means no static pods, which means no API server, which means a dead cluster.

Phase 6: The Real Fix — The Symlink

The solution was elegant: create the directory and symlink it back to our static resolv.conf:

# Create the directory kubelet expects
sudo mkdir -p /run/systemd/resolve

# Symlink to our static resolv.conf with real DNS servers
sudo ln -sf /etc/resolv.conf /run/systemd/resolve/resolv.conf

# Restart kubelet
sudo systemctl restart kubelet

Applied this to all three nodes. Within 30 seconds:

$ kubectl get nodes
NAME     STATUS   ROLES           AGE   VERSION
kube-1   Ready    control-plane   26h   v1.34.3
kube-2   Ready    control-plane   26h   v1.34.3
kube-3   Ready    <none>          25h   v1.34.3

The cluster was back!

Phase 7: The Final Run — 424/424 ✅

Third time's the charm. With the DNS properly configured and the kubelet symlink in place, I ran Sonobuoy one last time:

./sonobuoy run --mode=certified-conformance \
    --sonobuoy-image=sonobuoy/sonobuoy:v0.57 \
    --systemd-logs-image=sonobuoy/systemd-logs-arm64:v0.4 \
    --wait

~1 hour 42 minutes later:

Ran 424 of 7137 Specs in 6112.853 seconds
SUCCESS! -- 424 Passed | 0 Failed | 0 Pending | 6713 Skipped
PASS

ALL 424 TESTS PASSED. 🎉

The PR Submission

The Commit Squashing Lesson

The CNCF conformance repo has strict CI checks. One of them: exactly one commit per submission. My branch had accumulated multiple commits during the debugging process. The conformance bot flagged it:

[FAIL] it appears that there is not exactly one commit.
Please rebase and squash with git rebase -i HEAD

The fix:

# Reset to the base commit (keeping all changes staged)
git reset --soft <base-commit-hash>

# Create a single commit with DCO sign-off
git commit -s -m "Conformance results for v1.34/kubespray"

# Force push to update the PR
git push --force-with-lease origin conformance/v1.34-kubespray

DCO Sign-Off

The CNCF requires a Developer Certificate of Origin (DCO) sign-off on every commit. This is done with the -s flag in git commit, which adds:

Signed-off-by: Your Name <your-email@example.com>

The Result

After squashing into a single signed commit, the bot confirmed:

All requirements (15) have passed for the submission!

On February 10, 2026, Taylor Waggoner (CNCF staff) merged the PR with:

"You are now Certified Kubernetes 🎉"

PR #4078 — Merged.

What I Learned

Technical Skills

• Linux DNS internals: How systemd-resolved works, the difference between stub resolvers (127.0.0.53) and direct DNS, and the resolution chain

• Kubernetes node architecture: How kubelet starts static pods (API server, scheduler, controller-manager) using manifest files in /etc/kubernetes/manifests/

• Container networking: How CoreDNS resolves cluster DNS and how host DNS configuration affects pod DNS resolution

• Sonobuoy & e2e testing: Running conformance tests, interpreting results, understanding the difference between test failures and infrastructure failures

• ARM64 considerations: Not all CNI plugins support ARM64, Sonobuoy needs architecture-specific images, and some configurations behave differently on ARM

Open Source Best Practices

• Read existing submissions first: Looking at previous conformance PRs saved me hours

• Follow CI requirements precisely: Single commit, DCO sign-off, correct directory structure, and automation enforces these strictly

• Document everything: The README should contain reproducible steps so anyone can verify your results

• Be patient with debugging: What seemed like a DNS issue turned out to be a kubelet configuration dependency. Layer by layer, the picture became clear

The Debugging Mindset

The biggest lesson was systematic troubleshooting. When the tests failed:

1. Read the actual logs, not just the summary

2. Check each component independently (kubelet → containerd → API server → pods)

3. Understand the dependency chain (kubelet needs DNS config → DNS config path is hardcoded → removing systemd-resolved removed that path)

4. Test incrementally (verify DNS works before running full test suite)

The Complete DNS Fix

For anyone running Kubespray conformance tests on Ubuntu 24.04 (or any OS with systemd-resolved), here's the complete fix that works:

# On each node (kube-1, kube-2, kube-3):

# 1. Disable systemd-resolved
sudo systemctl stop systemd-resolved
sudo systemctl disable systemd-resolved

# 2. Set up direct DNS
sudo rm -f /etc/resolv.conf
echo -e 'nameserver 8.8.8.8\nnameserver 8.8.4.4' | sudo tee /etc/resolv.conf

# 3. Create the symlink kubelet expects (THIS IS THE KEY STEP)
sudo mkdir -p /run/systemd/resolve
sudo ln -sf /etc/resolv.conf /run/systemd/resolve/resolv.conf

# 4. Restart services
sudo systemctl restart containerd kubelet

Wait 2-3 minutes for the cluster to stabilize before running Sonobuoy.

Resources

• Kubespray Documentation

• CNCF Conformance Instructions

• Sonobuoy Documentation

If you're working on a conformance submission and hit DNS issues on Ubuntu/Fedora, the symlink fix above might save you a few hours. Happy testing!

Ever committed code with the wrong Git author? Whether you're juggling personal and work accounts or just forgot to configure your identity in a new project, this happens to the best of us. The good news? You can fix it safely and in this guide, we'll do it the smart way using a Docker sandbox plus real GitHub practice.

What You'll Learn

• Git rebase fundamentals (interactive mode explained simply)

• Three real-world scenarios for fixing wrong authors

• Safe Docker lab environment (no risk to your actual Git setup)

• Hands-on GitHub practice (see the fixes in action)

Why This Approach Works

Docker Container: Keeps your global Git config safe while you experiment
Real GitHub Repo: See the actual before/after results in GitHub's UI
Progressive Scenarios: From simple local fixes to complex merged commits

We'll simulate this common mistake:

• Global user (wrong) → surajdeveloper <surajdeveloper@example.com> → You have made commit with this

• Repo user (correct) → surajdevops <surajdevops@example.com> → But you should have made the commit with this.

Git Rebase : The Foundation

Before diving into fixes, let's understand why we use Git rebase for author corrections.

What is Git Rebase?

Takes your commits and replays them on top of another branch, creating a straight line history.

git rebase main

What it does:

• Moves your commits to the tip of main branch

• Creates new commit IDs (rewrites history)

• Makes history look like you worked on the latest code all along

What is Git Merge?

Git Merge = Combines two branches by creating a merge commit that joins them together.

git merge feature

What it does:

• Keeps original commits unchanged

• Creates one new "merge commit"

• Preserves the branching structure

When to Use?

• Merge: When working with others on the same branch

• Rebase: When cleaning up your local commits before opening a PR

Golden Rule: Never rebase shared branches!

Why Use Interactive Rebase for Fixes?

If you only need to fix the latest commit, you can use:

git commit --amend --author="Correct Name <correct@email.com>"

But if the wrong author is buried in the middle of history, you need to:

1. Stop at that specific commit

2. Fix the author information

3. Replay the remaining commits

That's exactly what git rebase -i (interactive rebase) does!

Rebase vs Merge: Quick Comparison

• Merge = Combines two branches while preserving original commit history

• Rebase = Replays commits on a new base, rewriting history for a clean linear timeline

Interactive Rebase: What Happens Under the Hood

When you run git rebase -i HEAD~3, here's the journey:

Visual: Before & After (Actual Git Output)

Before (wrong author on commits):

$ git log --oneline --graph --pretty=format:'%h - %an <%ae> - %s'
* 9a3f8d2 - surajdeveloper <surajdeveloper@example.com> - Add gitignore
* 7b2c4e1 - surajdeveloper <surajdeveloper@example.com> - Add main application file
* 5d1a3b9 - surajdevops <surajdevops@example.com> - Initial commit

Notice commits 9a3f8d2 and 7b2c4e1 have the wrong author (surajdeveloper).

During interactive rebase:

$ git rebase -i HEAD~3
# Editor opens showing:
pick 5d1a3b9 Initial commit
pick 7b2c4e1 Add main application file  
pick 9a3f8d2 Add gitignore

# You change it to:
pick 5d1a3b9 Initial commit
edit 7b2c4e1 Add main application file  # <- mark as edit
edit 9a3f8d2 Add gitignore              # <- mark as edit

After fixing with rebase + amend:

$ git log --oneline --graph --pretty=format:'%h - %an <%ae> - %s'
* 2f8a9c5 - surajdevops <surajdevops@example.com> - Add gitignore
* 4e7b3d1 - surajdevops <surajdevops@example.com> - Add main application file
* 5d1a3b9 - surajdevops <surajdevops@example.com> - Initial commit

✅ All commits now show the correct author (surajdevops)!
⚠️ Note: The commit hashes changed (9a3f8d2 → 2f8a9c5) because we rewrote history.

🐳 Setting Up Your Docker Git Lab: Let's do it hands-on

Let's create a completely isolated environment for safe experimentation.

Build & Run the Lab with the Docker image and clone the GitHub repo:

Image: https://hub.docker.com/r/surajkumar00/git-rebase-lab
GitHub repo: https://github.com/Suraj-kumar00/learn-git

Run interactive container

# Run interactive container
docker run -it --rm surajkumar00/git-rebase-lab

git clone https://github.com/Suraj-kumar00/learn-git.git
cd learn-git

You're now inside a clean Ubuntu container and GitHub repo with Git ready to go!

Lab Exercise: Creating the Problem

Let's simulate the real-world scenario where you accidentally use the wrong Git identity.

Step 0: Create a feature branch

git checkout -b feature

# Check the branch
git branch

Step 1: Set Wrong Global Identity

Inside the Docker container:

git config --global user.name "surajdeveloper"
git config --global user.email "surajdeveloper@example.com"

# Verify it's set
git config --global --list
# or
git config --list

Step 2: Let's make some Commits

echo "console.log('Hello World');" > app.js
git add app.js
git commit -m "Add main javascript application file"

echo "print('Hello World')" > main.py
git add main.py
git commit -m "Add main python application file"

echo "node_modules/" > .gitignore
git add .gitignore
git commit -m "Add gitignore"

Step 3: Check the Damage

git log

# or 

git log --oneline

# or

git log --pretty=format:"%h %an <%ae> %s"

Output shows all commits have the wrong author:

a1b2c3d surajdeveloper <surajdeveloper@example.com> Add gitignore
b4c5d6e surajdeveloper <surajdeveloper@example.com> Add main python application file  
c7d8e9f surajdeveloper <surajdeveloper@example.com> Add main javascript application file

Step 4: Set Correct Repo-Level Identity

git config user.name "surajdevops"
git config user.email "surajdevops@example.com"

# Verify repo config overrides global
git config user.name
git config user.email

New commits will now use the correct author, but what about the existing ones?

Scenario 1: Fix Local Commits (Before Push)

This is the safest scenario — commits exist only on your local machine.

Interactive Rebase in Action

# Start interactive rebase for last 3 commits
git rebase -i HEAD~3

This opens your editor with something like:

pick c7d8e9f Add main javascript application file
pick b4c5d6e Add main python application file
pick a1b2c3d Add gitignore

# Commands:
# p, pick <commit> = use commit
# e, edit <commit> = use commit, but stop for amending
# r, reword <commit> = use commit, but edit the commit message

Mark Commits for Editing

Change pick to edit for commits you want to fix:

edit c7d8e9f Add main javascript application file
edit b4c5d6e Add main python application file
edit a1b2c3d Add gitignore

Save and exit (in vim: esc shift+: press x)

Fix Each Commit

Git will stop at each commit marked for editing:

# When Git pauses at first commit
git commit --amend --author="surajdevops <surajdevops@example.com>" --no-edit

# Continue to next commit
git rebase --continue

# Repeat for each commit Git stops at

Verify the Fix

git log --pretty=format:"%h %an <%ae> %s"

Now all commits show the correct author! ✅

Scenario 1.1: Fix Initial Commit (Including Root)

Special case: What if the initial commit also has the wrong author? Regular HEAD~N won't include the root commit.

The Problem: Initial Commit Has Wrong Author

git log --pretty=format:"%h %an <%ae> %s"

Output shows:

a1b2c3d surajdeveloper <surajdeveloper@example.com> Add gitignore
b4c5d6e surajdeveloper <surajdeveloper@example.com> Add main python application file  
c7d8e9f surajdeveloper <surajdeveloper@example.com> Add main javascript application file
d8e9f0g surajdeveloper <surajdeveloper@example.com> Initial commit  ← WRONG!

The Solution: Use --root Flag

# Include the initial commit in interactive rebase
git rebase -i --root

This opens the editor with ALL commits including the root:

pick d8e9f0g Initial commit          ← Root commit included!
pick c7d8e9f Add main javascript application file
pick b4c5d6e Add main python application file
pick a1b2c3d Add gitignore

Mark Commits (Including Root) for Editing

edit d8e9f0g Initial commit          ← Fix the root!
edit c7d8e9f Add main javascript application file
edit b4c5d6e Add main python application file
edit a1b2c3d Add gitignore

Fix Each Commit (Same Process)

# Git stops at root commit first
git commit --amend --author="surajdevops <surajdevops@example.com>" --no-edit
git rebase --continue

# Continue for each subsequent commit...

Why --root is Essential

• Regular rebase (HEAD~4) excludes the initial commit

• Root rebase (--root) includes everything from the beginning

• This is the only way to edit the initial commit's author

Scenario 2: Wrong Author Already Pushed (Branch Not Merged)

If you've already pushed the commits to a feature branch (but not merged to main), you can still fix them.

Setup: Push to Remote Branch

First, let's simulate this scenario:

# Create a feature branch (if not already on one)
git checkout -b feature/user-auth

# Push to remote (if you have access to push)
git push -u origin feature/user-auth

Fix and Force Push

After fixing the commits locally (using the same rebase process above):

# Force push the corrected history
git push origin feature/user-auth --force

⚠️ Important Warnings:

• Only force push if you own the branch

• Coordinate with your team before rewriting pushed history

• Never force push to main or master without team agreement

Scenario 3: Wrong Author Already Merged to Main

This is the most complex scenario. Once commits are merged into the main branch, rewriting history becomes dangerous.

Option 1: Accept and Document (✅ Recommended)

Best practice → Don't rewrite main branch history. Instead, add documentation:

git commit --allow-empty -m "docs: commits abc123-def456 were authored by surajdevops, not surajdeveloper"

Option 2: Rewrite Main (⚠️ Dangerous)

Only do this if:

• Your entire team agrees

• You can coordinate a synchronized reset for all team members

• The repository is not public or widely used

# Fix commits using rebase
git rebase -i HEAD~5  # Adjust number as needed

# Force push main (DANGEROUS!)
git push origin main --force

# All team members must then:
# git fetch origin
# git reset --hard origin/main

Advanced Tips & Best Practices

For this you can read my this blog Managing Multiple GitHub/Git Accounts on One Machine (Personal + Work)

Prevent Future Mistakes

1. Set repo-specific config immediately in new projects:

# Right after git clone or git init
git config user.name "Your Correct Name"
git config user.email "your.correct@email.com"

2. Create Git aliases for quick identity switching:

# Add to ~/.gitconfig
git config --global alias.work 'config user.email "work@company.com"'
git config --global alias.personal 'config user.email "personal@gmail.com"'

# Usage: git work or git personal

When NOT to Rewrite History

Never rewrite history when:

• ❌ Commits already merged to main/master

• ❌ Public repositories with external contributors

• ❌ Other team members have based work on your commits

• ❌ CI/CD systems have already processed the commits

• ❌ The commits have been tagged for a release

Quick Reference: Rebase Commands

# Fix last commit only
git commit --amend --author="Name <email>"

# Fix multiple recent commits
git rebase -i HEAD~N  # N = number of commits

# Fix commits including the initial commit
git rebase -i --root

# Fix commits after specific commit
git rebase -i abc123^  # abc123 = commit hash

# Abort rebase if something goes wrong
git rebase --abort

# Skip a problematic commit during rebase
git rebase --skip

Complete Setup Commands Summary

# 1. Run the Docker lab
docker run -it --rm surajkumar00/git-rebase-lab

# 2. Clone the practice repo
git clone https://github.com/Suraj-kumar00/learn-git.git
cd learn-git

# 3. Create feature branch
git checkout -b feature

# 4. Set wrong identity and create commits
git config --global user.name "surajdeveloper"
git config --global user.email "surajdeveloper@example.com"

# 5. Make commits (will have wrong author)
echo "console.log('Hello World');" > app.js && git add app.js && git commit -m "Add main javascript application file"
echo "print('Hello World')" > main.py && git add main.py && git commit -m "Add main python application file"
echo "node_modules/" > .gitignore && git add .gitignore && git commit -m "Add gitignore"

# 6. Set correct identity
git config user.name "surajdevops"
git config user.email "surajdevops@example.com"

# 7. Fix the commits

# or git rebase -i --root for initial commit
git rebase -i HEAD~3 

# Change 'pick' to 'edit', then for each commit:
git commit --amend --author="surajdevops <surajdevops@example.com>" --no-edit
git rebase --continue

# 8. Verify the fix
git log --pretty=format:"%h %an <%ae> %s"

Your Next Steps

1. Configure proper Git identities in all your current projects to prevent future mistakes

2. Share this guide with your team to prevent future author mix-ups

Remember: Git is powerful, but with great power comes great responsibility hehe. Always practice in safe environments before applying these techniques to important repositories!

Happy rebasing!

Found this helpful? Star the practice repo and share with your team! Questions or improvements? Open an issue on the above GitHub repo.

NGINX has revolutionized modern web infrastructure, becoming the backbone of high-performance applications worldwide. In this part 1 you'll learn about NGINX fundamentals configurations, theoretical knowledge with real-time scenario examples.

Originally developed by Igor Sysoev in 2004, NGINX was created to solve the infamous C10K problem and has since evolved into one of the most powerful and widely adopted web servers in the world.

What is Forward Proxy and Reverse Proxy?

Understanding proxy servers is fundamental to grasping NGINX's core functionality, as it excels primarily as a reverse proxy server.

Forward Proxy:

A forward proxy acts as an intermediary between clients and the internet, sitting on the client side of the network.

Key Characteristics:

• Acts on behalf of the client

• Sits between client and the public internet

• Forwards client requests to servers

• Server doesn't know which specific client made the request

• Primarily serves client needs

Real-World Example: Corporate networks use forward proxies to:

• Filter employee internet access

• Block social media and non-work websites

• Cache frequently accessed content to save bandwidth

• Provide anonymity for internal users

• Monitor and log internet usage

Reverse Proxy:

A reverse proxy sits between clients and backend servers, acting on behalf of the server infrastructure.

Key Characteristics:

• Acts on behalf of the server

• Sits between internet clients and backend servers

• Hides server implementation details from clients

• Distributes incoming requests across multiple backend servers

• Provides additional services like SSL termination, caching, and load balancing

Real-World Example: Netflix, Amazon, and Google use reverse proxies to:

• Distribute user requests across thousands of servers worldwide

• Cache popular content closer to users

• Terminate SSL connections at the edge

• Provide DDoS protection and security filtering

• Ensure high availability and fault tolerance

Understanding NGINX: Architecture & Use Cases

What is NGINX?

NGINX (pronounced "engine-x") is a widely used open-source tool that does much more than just serve web pages. Known for its speed and reliability, it also works as a reverse proxy, load balancer, and caching server. Whether you're streaming media, handling email protocols like SMTP or IMAP, or routing HTTP and TCP traffic, NGINX is built to handle it all with efficiency.

Core Capabilities:

• Web Server: Serving static and dynamic content with minimal resource usage

• Reverse Proxy: Forwarding client requests to backend application servers

• Load Balancer: Distributing incoming traffic across multiple backend servers

• HTTP Cache: Storing frequently requested content to reduce backend load

• SSL/TLS Termination: Handling encryption/decryption at the network edge

• Mail Proxy: Managing SMTP, POP3, and IMAP protocol connections

• Stream Proxy: Handling TCP and UDP traffic for various applications

What problem it solves?

NGINX was specifically designed to solve the C10K problem - the challenge of handling 10,000 (or more) concurrent client connections on a web server efficiently.

The Traditional Problem: Before NGINX, traditional web servers like Apache used a process-per-connection or thread-per-connection model:

This approach became unsustainable as:

• Each connection consumed significant memory (8-12MB per process)

• Context switching between processes became expensive

• System resources were quickly exhausted

• Performance degraded dramatically under high load

NGINX's Solution: NGINX was built to solve this problem using an asynchronous, event-driven model, making it lightweight and able to handle tens of thousands of connections simultaneously efficiently and reliably.

Why NGINX Became Essential:

The traditional web servers like Apache used a process-per-connection model, which became inefficient as web traffic grew exponentially. Each connection required a separate process or thread, consuming significant memory and CPU resources. NGINX's innovative approach changed this paradigm entirely.

NGINX Architecture: The Process Model

NGINX uses an event-driven, asynchronous architecture that sets it apart from traditional web servers and consists of several components such as**:**

1. Master process – Controls the main NGINX instance. It manages configuration, and is responsible for starting, stopping, and supervising the worker processes.

2. Worker processes – Handle all the actual work: managing client connections, serving static content, proxying requests, load balancing, and SSL/TLS termination.

3. Cache loader – Runs at startup to load cache metadata from disk into memory, making cached content immediately available after NGINX boots.

4. Cache manager – Runs in the background at intervals to check the cache directory, remove expired data, and ensure disk usage stays within limits.

5. Shared memory – Provides inter-process communication and storage for shared state, such as cache indexes, rate limiting counters, and load-balancing information.

Why This Architecture Matters:

• Memory Efficiency: One worker can handle thousands of connections with minimal memory overhead

• CPU Efficiency: No context switching between processes for each request

• Scalability: Performance degrades gracefully under high load

• Stability: If a worker crashes, the master spawns a new one without affecting other connections

• Resource Optimization: Efficient use of system resources leads to better overall performance

Core Use Cases

NGINX's versatility makes it suitable for numerous deployment scenarios:

1. Web Server: Serving static content (HTML, CSS, JS, images) with minimal overhead

2. Reverse Proxy: Forwarding requests to backend applications

NGINX Alternatives

While NGINX is widely used, a few alternatives are worth knowing:

1. Apache HTTP Server – A long-standing web server with strong legacy support and a rich module ecosystem, but less efficient under heavy load compared to NGINX.

2. HAProxy – Specializes in high-performance load balancing and traffic distribution. Great for reliability, but not designed to serve static content.

3. Traefik – A modern, cloud-native reverse proxy with built-in support for containers, service discovery, and automatic SSL management.

4. Cloudflare (as a Service) – A managed CDN and security platform offering DDoS protection, WAF, and global content delivery but relies on a third-party provider.

Setting Up Nginx with Docker: Hands-On

Prerequisites

Before we begin, ensure you have Docker installed on your system. You can download it from Docker's official website.

Step 1: Setting Up the Docker Container

Let's start by creating and running an Ubuntu container with Nginx:

docker run -it --name nginx-docker -p 8080:80 ubuntu

Command Breakdown:

• -name nginx-docker → Assigns a custom name to your container

• p 8080:80 → Maps your host machine's port 8080 to the container's port 80

• it ubuntu → Creates an interactive terminal session with Ubuntu

This command will download the Ubuntu image (if not already present) and start an interactive container session.

Step 2: Installing Nginx and Essential Tools

Once inside the container, update the package list and install Nginx:

# Update package repositories
apt update

# Install Nginx web server
apt install nginx -y

# Install vim text editor (useful for editing config files)
apt install vim -y

After running the apt install vim -y command first choose number 5 which is Asia and then choose 44 It’s Kolkata, we are choosing these to install IST timezone while installing the vim tool.

Pro Tip: Always run apt update first to ensure you're installing the latest versions of packages.

Step 3: Verify Nginx Installation

Check if Nginx was installed successfully:

nginx -v
or 
nginx -V

You should see output similar to: nginx version: nginx/1.18.0 (Ubuntu)

Step 4: Understanding Nginx Directory Structure

After installation, all Nginx files are stored in /etc/nginx. Let's explore this directory:

cd /etc/nginx
ls -la

You'll see a structure like this:

drwxr-xr-x 8 root root 4096 Aug 19 13:38 ./
drwxr-xr-x 1 root root 4096 Aug 19 13:36 ../
drwxr-xr-x 2 root root 4096 May 27 10:28 conf.d/
-rw-r--r-- 1 root root 1125 Dec  1  2023 fastcgi.conf
-rw-r--r-- 1 root root 1055 Dec  1  2023 fastcgi_params
-rw-r--r-- 1 root root 5465 Dec  1  2023 mime.types
drwxr-xr-x 2 root root 4096 May 27 10:28 modules-available/
drwxr-xr-x 2 root root 4096 May 27 10:28 modules-enabled/
-rw-r--r-- 1 root root 1446 Aug 19 13:37 nginx.conf
-rw-r--r-- 1 root root  636 Dec  1  2023 scgi_params
drwxr-xr-x 2 root root 4096 Aug 19 13:24 sites-available/
drwxr-xr-x 2 root root 4096 Aug 19 13:24 sites-enabled/
drwxr-xr-x 2 root root 4096 Aug 19 13:24 snippets/

Key Files and Directories:

• nginx.conf → Main configuration file (most important!)

• sites-available/ → Contains individual site configurations

• sites-enabled/ → Contains symlinks to active site configurations

• conf.d/ → Additional configuration files

Step 5: Managing Nginx Service

Starting Nginx

To start the Nginx service:

service nginx start

When to use: After installation or when Nginx has been stopped.

Checking Nginx Status

To verify Nginx is running:

service nginx status

Alternative method to check running processes:

ps aux | grep nginx

You should see output showing:

• nginx: master process (main Nginx process)

• nginx: worker process (handles actual requests)

Stopping Nginx

When you need to stop Nginx:

service nginx stop

When to use: During maintenance, troubleshooting, or when shutting down your server.

Step 6: Testing Your Nginx Installation

Since we mapped port 80 to 8080, open your web browser and navigate to:

http://localhost:8080

You should see the default Nginx Welcome Page!

Important Note: Nginx runs on port 80 by default, but since we're using Docker, we've mapped container port 80 to host port 8080. Make sure no other service is using port 8080 on your host machine.

Step 7: Creating a Custom Configuration

Now let's customize Nginx with our own configuration.

Backup the Original Configuration

Always backup before making changes:

cd /etc/nginx
mv nginx.conf nginx-backup.conf

Create New Configuration File

You can either create an empty file first or directly edit:

# Option 1: Create empty file then edit
touch nginx.conf
vim nginx.conf

# Option 2: Directly create and edit
vim nginx.conf

Add Custom Configuration

Insert the following content into your new nginx.conf:

events {
    worker_connections 1024;
}

http {
    server {
        listen 80;
        server_name _;

        location / {
            return 200 "Hello from Nginx Custom Configuration via Docker\\n";
            add_header Content-Type text/plain;
        }
    }
}

How to exit the vim editor: After pasting this, press esc and then shift + : and the press x and press enter

Configuration Breakdown:

• events → Defines connection processing parameters

• worker_connections 1024 → Maximum connections per worker process

• http → Main HTTP context

• listen 80 → Port Nginx listens on

• server_name _ → Catch-all server name

• location / → Handles all requests to root path

• return 200 → Returns HTTP 200 status with custom message

Step 8: Testing and Reloading Configuration

Test Configuration Syntax

Before applying changes, always test the configuration:

nginx -t

You should see something like this:

Reload Configuration

If the test passes, reload Nginx to apply changes:

nginx -s reload

Why reload instead of restart?

• reload → Applies new configuration without dropping existing connections

• restart → Stops and starts Nginx, dropping all connections

Verify Your Changes

Visit http://localhost:8080 again. You should now see your custom message: "Hello from Nginx Custom Configuration via Docker"

Common Commands Summary

Here's a quick reference of essential Nginx commands:

# Service management
service nginx start     # Start Nginx
service nginx stop      # Stop Nginx
service nginx status    # Check status
service nginx restart   # Full restart

# Configuration management
nginx -t               # Test configuration
nginx -s reload        # Reload configuration
nginx -s stop          # Graceful stop
nginx -s quit          # Graceful shutdown

# Information
nginx -v              # Show version
nginx -V              # Show version and compile options

In the next parts of this nignx blogs we’ll learn:

Now that you have a basic Nginx setup running, you can explore:

• Serving static files

• Setting up reverse proxy

• SSL/TLS configuration

• Load balancing

• Custom error pages

Conclusion

You've successfully set up Nginx in a Docker container, learned how to manage the service, and created your first custom configuration. This foundation will serve you well as you continue your journey with web servers and containerization.

Remember: Always test your configurations before applying them, and keep backups of working configurations. Happy learning!

Thanks for reading…

A practical, real-world guide to understanding and applying Git cherry-pick with actual developer mistakes and fixes.

What is git cherry-pick?

git cherry-pick lets you selectively apply commits from one branch to another — like copying specific code changes without merging entire branches.

Now let’s understand a quick comparison cherry-pick vs merge vs rebase

TL;DR:

• Use cherry-pick for precision, specific commits only

• Use merge to combine full branch history

• Use rebase to clean history and avoid unnecessary merges

Why You Might Need cherry-pick

1. You committed on the wrong branch (happens more than we admit)

2. You need a bug fix from one branch to another

3. You want to copy only certain commits, not merge everything

4. You made changes, reverted them locally, but commits got mixed up

My Real-Time Scenario: How I Discovered cherry-pick

Let me share how I personally ran into this:

I made some changes in a branch (let's call it branch A) but accidentally reverted the commits locally, leaving them in the unstaged area. Later, I re-committed those changes — but this time, I was working on branch B, where I only wanted my latest changes, not the leftover ones from branch A.

I pushed the branch to GitHub, went to open the PR, and BOOM — I saw two commits:

• One commit with my intended change for branch B

• But another unintended commit sneakily came from branch A

Total mess, right? But no worries — I fixed this cleanly using git cherry-pick.

I removed the wrong commit from branch B, stayed on branch A, cherry-picked only the commit I actually needed onto branch B, and pushed it again. Clean history, problem solved.

Step-by-Step Practical Guide to git cherry-pick

1. View Commit History with git log

git log --oneline --graph --all

✅ This shows a simple, visual commit history across all branches.

Tip: If you're working on a shared repository with your team and didn't create your own branch yet, you can check history of a specific branch like:

git log <branch-name> --oneline --graph --all

Example:

git log main --oneline --graph --all

✅ Use this to find the commit hash (abc123) you want to cherry-pick.

✅ Copy that hash — you'll need it in the next step.

2. Switch to Your Target Branch

Before you apply the commit, make sure you're on the correct branch:

git status

This shows your current branch.

If unsure, list all branches:

git branch

Switch to your target branch:

git checkout <your-branch-name>

Example:

git checkout feature/login

3. Cherry-Pick the Commit

git cherry-pick <commit-hash>

✅ This applies only the specific commit to your current branch, without merging the entire source branch or bringing unrelated changes.

Example:

git cherry-pick abcd1234

If the commit applies cleanly, you're done. If there's a conflict, Git will pause and you resolve it manually.

Cherry-Pick Multiple Commits

One by one:

git cherry-pick <commit1> <commit2>

Range of commits:

git cherry-pick commitA^..commitB

Handling Conflicts During Cherry-Pick

Conflict? No stress. Git pauses:

CONFLICT (content): Merge conflict in file.js

Resolve it:

git add .
git cherry-pick --continue

Or abort:

git cherry-pick --abort

You can also follow this short youtube video if you want:

Final Thoughts

Mistakes happen — commits go to the wrong place, branches get mixed up.

git cherry-pick is the clean, targeted way to:

✅ Move specific commits

✅ Fix accidental commits

✅ Avoid messy merges

Once you try it practically, you’ll use it confidently — like I did when I accidentally messed up branches.

Thanks for reading guys…

In this guide I’ll walks you through setting up two GitHub accounts securely, cleanly, and with verified SSH-signed commits all on a single machine.

Objectives

1. Work with both personal and work GitHub repositories (public & private).

2. Use two separate GitHub accounts on one machine.

3. Ensure verified commits via SSH commit signing.

4. Maintain a scalable, secure, and professional Git setup.

Step 1: Generate Separate SSH Keys

Create separate SSH key pairs for each GitHub account:

You can name these according to your preferences while generating the SSH keys

• for-personal

• for-work

# Personal Key
ssh-keygen -t ed25519 -C "you.email@gmail.com" -f ~/.ssh/for-personal

# Work Key
ssh-keygen -t ed25519 -C "you.workemail@gmail.com" -f ~/.ssh/for-work

Do not overwrite the default id_ed25519. Keeping keys separate ensures flexibility and security.

Step 2: Add Keys to SSH Agent

# View loaded keys
ssh-add -l

# Add new keys
ssh-add ~/.ssh/for-personal
ssh-add ~/.ssh/for-work

# Remove all existing keys (optional reset)
ssh-add -D

Step 3: Configure SSH config File

Create or edit your SSH config file:

touch ~/.ssh/config

Add the following content:

# Global settings
Host *
  AddKeysToAgent yes
  UseKeychain yes
  IdentitiesOnly yes
  ServerAliveInterval 60
  ServerAliveCountMax 3

# Personal GitHub
Host personal.github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/for-personal

# Work GitHub
Host work.github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/for-work

Step 4: Add Public Keys to GitHub

Upload the corresponding .pub files to each account:

• ~/.ssh/for-personal.pub → Personal GitHub

• ~/.ssh/for-work.pub → Work GitHub

GitHub → Settings → SSH and GPG Keys → New SSH Key

Step 5: Clone Repositories Using SSH Host Aliases

# Clone personal repo
git clone git@personal.github.com:your-username/my-repo.git

# Clone work repo
git clone git@work.github.com:my-org/work-repo.git

Step 6: Configure Git Identity per Repository

Set identity locally inside each project to avoid global conflicts:

# Inside Personal Repo
cd ~/PersonalRepo

git config user.name "your name"
git config user.email "your email"

# Inside Work Repo
cd ~/WorkRepo

git config user.name "your name"
git config user.email "your work email"

Step 7: Enable Verified Commits with SSH Signing

GitHub supports SSH-based commit signing, separate from SSH authentication.

Generate Signing Keys

# Personal Signing Key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_signing_personal -C "signing-personal"

# Work Signing Key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_signing_work -C "signing-work"

Add Signing Keys to GitHub

Go to GitHub → Settings → SSH and GPG Keys → New Signing Key and paste the contents of each .pub file.

Step 8: Configure Git to Use Signing Keys

Set up commit signing in each repo:

# Personal Repo
cd ~/PersonalRepo

git config commit.gpgsign true
git config gpg.format ssh
git config user.signingkey ~/.ssh/id_ed25519_signing_personal.pub

# Work Repo
cd ~/WorkRepo

git config commit.gpgsign true
git config gpg.format ssh
git config user.signingkey ~/.ssh/id_ed25519_signing_work.pub

Pre-Commit Workflow: Start SSH Agent and Test Connections

Before making commits in any repository, ensure your SSH agent is running and keys are loaded:

# Ensure SSH agent is running and keys are loaded
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/personal_github
ssh-add ~/.ssh/work_github

# Test connections to verify which account will be used
ssh -T git@personal.github.com
# or
ssh -T git@work.github.com

# Now proceed with your git operations
git add .
git commit -m "Your commit message"
git push origin main

Successful messages like Hi broh! You've successfully authenticated confirm proper setup.

Note: This step is particularly important after system restarts or when opening new terminal sessions, as the SSH agent may not be running or may not have your keys loaded.

Final Result

• Secure and clean GitHub SSH setup for both accounts.

• Verified commits using SSH signing keys.

• Separate identities per project.

Bonus Tip

To view signed commits:

git log --show-signature

The end! - Thanks for reading…

Docker is a powerful containerization platform that streamlines application development, deployment, and scaling of containerized applications. In this blog I’ll covers important Docker concepts like networking, Volumes, writing compose file in docker, difference in Docker and Kubernetes, Multi-Stage build in dockerfile with practical insights for developers and DevOps engineers.

Docker Networking

Docker Networking enables communication between containers, the Docker host, and the external world. When a container is running, it is attached to a virtual network. This network determines how the container can talk to other containers, services, or the internet. Docker uses network drivers to abstract different networking strategies. Understanding these drivers helps optimize performance, security, and communication within your containerized environments.

Docker provides three primary network drivers but there are more according to the usecases:

Bridge (Default Driver)

If you're running containers on a single host and want them to talk to each other, Bridge is your go-to.

How it works: Creates an internal network and assigns IP addresses to containers.

Use case:
You’re running a Node.js API and a MongoDB container on your laptop. Both need to talk to each other. Use the default bridge network.

Practical Example:

docker network create --driver bridge my_bridge
docker run -d --name db --network my_bridge mongo
docker run -d --name api --network my_bridge node-app

api can now connect to MongoDB using mongodb://db:27017.

So see which containers are connected with the same network on bridge driver use this command:

# To see the detailed information on one or more networks
docker network inspect bridge

Host

Remove network isolation between the container and the Docker host, and use the host's networking directly.

How it works:

• The container shares the host’s network stack — it has no separate IP.

• Ports on the container are directly exposed on the host.

Use case:
Running an Nginx container on port 80 without port mapping, for better performance.

Practical Example:

docker run --rm --network host nginx

Nginx is now accessible directly at http://localhost without needing -p 80:80.

The flag --rm means:

Automatically remove the container when it exits.

None

You want a container completely cut off from all networks? Use this.

How it works:

• No network interface is created.

• Only useful for extreme isolation or containers communicating via files/mounted volumes only.

Use case:
Highly secure containers for background data processing with no need for communication.

Practical Example:

docker run --network none your_image

Want to learn more network drivers you can follow the official documentation of docker itself.

Link: https://docs.docker.com/engine/network/drivers

Difference Between Bridge and Host Network

How To Create Custom Networks In Docker

When you use Docker, containers by default get attached to the bridge network unless you specify otherwise. But for more control, better architecture, and easier service-to-service communication, custom networks are the way to go.

Practical Example:

# Step 1: Create a custom network
docker network create my_custom_net

# Step 2: Run Redis container on that network
docker run -d --name redis --network my_custom_net redis

# Step 3: Run backend app on the same network
docker run -d --name backend --network my_custom_net myapp

Now the backend container can reach the redis container using:

redis:6379

No need to figure out IP addresses. Docker’s DNS handles the resolution automatically.

Benefits of creating your own network

• Automatic container name resolution

• Better network isolation

• Cleaner communication between services

Docker Volumes

By default, Docker containers are ephemeral — meaning, when a container is deleted, all the data inside it is also lost. This is a big problem for things like:

• Databases (e.g., PostgreSQL, MongoDB)

• Log files

• User-generated content (uploads, configs)

To solve this, Docker provides volumes, which are persistent storage locations managed by Docker.

How to Use Docker Volumes

# Step 1: Create a Docker volume
docker volume create my_data

# Step 2: Mount the volume into a container
docker run -v my_data:/data my_image

This mounts the my_data volume into the container’s /data directory.

Benefits of Docker Volumes

• Persistence
  Data remains intact even if the container is removed, restarted, or recreated.

• Managed by Docker
  Volumes live outside the container filesystem and are managed in a secure and efficient way by Docker.

• Decouples Data from Containers
  You can attach the same volume to different containers (great for backup jobs, logs, etc.)

• Supports Named Volumes and Bind Mounts

  • Named volumes: managed by Docker (my_data)

  • Bind mounts: link to a host path (/home/user/data:/data)

Efficient Caching in Layers

Every command in your Dockerfile creates a layer. These layers are cached and reused by Docker to avoid rebuilding the entire image every time — this is known as layer caching.

A well-structured Dockerfile improves build time significantly. Why? Because Docker will reuse unchanged layers from cache instead of rebuilding them.

You’ll much understand Via this example:

Here we have the sample dockerfile and below I have given you a diagram in which you can see arranging the layers carefully can save the time everytime you build your dockerfile but in the first build it takes time much longer then the rebuild you do further.

# Build dependencies using lightweight Node.js base image
FROM node:18-alpine

# Set the working directory inside the container
WORKDIR /app

# Only copy dependency descriptor files (changes infrequently)
COPY package*.json ./

# Install only production dependencies (skips devDependencies)
RUN npm ci --only=production

# Copy the rest of the application source code (changes frequently)
COPY . .

# Define the command to run the app
CMD ["npm", "start"]

When you build a Docker image, each line in your Dockerfile creates a layer. Docker tries to reuse (cache) these layers in future builds to save time — especially when files or steps haven’t changed.

Benefits of Efficient Layer Caching

• Faster builds — Only changed layers are rebuilt.

• Smaller image layers — Cached intelligently.

• Repeatable builds — Fewer surprises and cleaner CI/CD.

Docker Multi-Stage Builds

Multi-stage builds allow you to use multiple FROM instructions in one Dockerfile. This helps you separate the build environment (where dependencies and tools are needed) from the production image (where only final code/artifacts go).

This results in:

• Smaller images 📦

• More secure builds 🔐

• Faster CI/CD deployment ⚡

So here I’m showing you one of my project’s multi-stage build Dockerfile

# Stage 1: Builder with full Node.js environment
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files first for optimal caching
COPY package.json package-lock.json ./

# Install all dependencies (including devDependencies)
RUN npm ci --no-audit --prefer-offline

# Copy source files
COPY . .

# Build the application
RUN npm run build

# Stage 2: Production image
FROM node:18-alpine

WORKDIR /app

# Copy built assets and dependencies
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json

# expose the port
EXPOSE 3000

# Start the application
CMD ["npm", "start"]

Best practices to write a good multi-stage dockerfile

• Use small base images like node:18-alpine or nginx:alpine.

• Name your stages (AS builder) for readability.

• Avoid copying unnecessary files (use .dockerignore).

• Keep the final image minimal (no build tools, dev dependencies).

• Use multi-stage even for simple projects to keep the image lean and clean.

Docker Compose

Docker Compose is a tool used to define and run multi-container Docker applications using a YAML file (docker-compose.yml or compose.yml or docker-compose.yaml). You describe the services, their images/build context, ports, environment variables, volumes, and networks — all in one place.

Understanding with Architecture diagram:

Consider an application split into a frontend web application and a backend service.

The frontend is configured at runtime with an HTTP configuration file managed by infrastructure, providing an external domain name, and an HTTPS server certificate injected by the platform's secured secret store.

The backend stores data in a persistent volume.

Both services communicate with each other on an isolated back-tier network, while the frontend is also connected to a front-tier network and exposes port 443 for external usage.

This is how you write a docker-compose.yml file:

version: '3.8'  # Specify Compose file version for compatibility

services:
  frontend:
    image: example/webapp                    # Frontend web application image
    ports:
      - "443:8043"                          # Map host port 443 to container port 8043 for HTTPS access
    networks:
      - front-tier                         # Connect to front-tier network (public-facing)
      - back-tier                          # Connect to back-tier network to talk to backend securely
    configs:
      - source: httpd-config               # Use external Apache HTTPD config
        target: /etc/httpd/conf/httpd.conf
    secrets:
      - server-certificate                 # Use external TLS/SSL certificate secret

  backend:
    image: example/database                 # Backend database service image
    volumes:
      - db-data:/etc/data                   # Persist database data in a named volume
    networks:
      - back-tier                         # Only expose backend internally on back-tier network

volumes:
  db-data:
    driver: flocker                        # Volume driver for data persistence
    driver_opts:
      size: "10GiB"                       # Allocate 10GB storage for DB data volume

configs:
  httpd-config:
    external: true                        # Reference an externally created config

secrets:
  server-certificate:
    external: true                       # Reference an externally managed secret (TLS cert)

networks:
  front-tier: {}                         # Define front-tier network (default bridge network)
  back-tier: {}                          # Define back-tier network (default bridge network)

Key commands

To start all the services defined in your compose.yaml or docker-compose.yml file:

docker compose up

To run the services in the background:

docker compose up -d

To rebuild Docker images and start containers:

docker compose up --build

To run the services in the background:

docker compose up --build -d

To stop and remove the running services:

docker compose down

If you want to monitor the output of your running containers and debug issues, you can view the logs with:

docker compose logs

To list all the services along with their current status:

docker compose ps
# or
docker ps

To list all the services along with stopped/exited:

docker compose ps -a
# or
docker ps -a

To watch the docker services current status continuously:

watch docker ps

To access shell inside a running container:

docker compose exec -it <container-id or container-name> sh
# or
docker compose exec -it <container-id or container-name> bash

Docker Swarm

Docker Swarm is Docker’s built-in clustering and orchestration tool. It allows you to combine multiple Docker hosts (nodes) into a single cluster called a swarm, making it easier to deploy and manage containerized applications at scale.

How it works:

• Cluster Setup: You start by initializing a swarm on one machine, which becomes the manager node.

• Node Management: Other Docker hosts can join this swarm as worker nodes.

• Service Deployment: You deploy your application as a service instead of individual containers.

• Scaling: You specify how many replicas (instances) of a service you want, and Swarm handles scheduling those across available nodes.

• Load Balancing: Swarm automatically distributes incoming requests among service replicas, balancing load and improving availability.

Steps to do to work with swarm:

docker swarm init

after initializing you’ll see this message:

Swarm initialized: current node (7ns53yw3fvsy1bvoma6tkyf8v) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-5r4ya9ic9x3rducjym1i4dxgg01rtydnmtuh9bq2k2khllnyt4-0jbi2g5s5kzon0drl65pux8jc 192.168.65.3:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

• Initializes the current Docker host as a swarm manager node.

• This sets up the cluster control plane.

docker service create --replicas 3 --name web nginx

After running above command you’ll this:

tzgrg9wjm0xunznt04zti2qd4
overall progress: 3 out of 3 tasks 
1/3: running   [==================================================>] 
2/3: running   [==================================================>] 
3/3: running   [==================================================>] 
verify: Service tzgrg9wjm0xunznt04zti2qd4 converged

• Creates a new service named web using the nginx image.

• Runs 3 replicas of the nginx container distributed across the swarm nodes.

• Swarm automatically schedules these replicas on different nodes (if available) and manages their lifecycle.

docker ps

After running the dcoker ps command you’ll these logs:

CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS     NAMES
2069f48eecbd   nginx:latest   "/docker-entrypoint.…"   16 seconds ago   Up 15 seconds   80/tcp    web.3.m3mtwuonvph1krfn0uvxe325r
9d52e9929221   nginx:latest   "/docker-entrypoint.…"   16 seconds ago   Up 15 seconds   80/tcp    web.2.5xzbt400wks7f7j55lqxi5ahe
4066263b7592   nginx:latest   "/docker-entrypoint.…"   16 seconds ago   Up 15 seconds   80/tcp    web.1.tv76fhg177h0zyo3zvf74gaii

Why use Docker Swarm?

• Simple to set up: Native Docker CLI commands manage the cluster.

• Built-in load balancing and scaling.

• High availability: Services automatically recover from node failures.

• Declarative service model: Define how many replicas and Docker manages the rest.

To clean up everything you've done with Docker Swarm, you need to:

1. Remove Services

First, remove any running services you created in the swarm:

docker service ls                 # List all services
docker service rm <service-name> # Remove each service

2. Remove the Swarm (Leave the Cluster)

If you're on a worker node, leave the swarm with:

docker swarm leave

If you're on the manager node, force it to leave (and dissolve the swarm):

docker swarm leave --force

Docker Vs Kubernetes

Docker:

Docker operates at the individual container level on a single operating system host. You must manually manage each host and setting up networks, security policies, and storage for multiple related containers can be complex.

Kubernetes:

Kubernetes operates at the cluster level. It manages multiple containerized applications across multiple hosts, providing automation for tasks like load balancing, scaling, and ensuring the desired state of applications.

In short, Docker focuses on containerization and running containers on individual hosts, while Kubernetes specializes in managing and orchestrating containers at scale across a cluster of hosts.

With this diagram you’ll understand what we have talked above.

Wrapping up with Demo:

If you want to try out yourself this demo you can use this project:

So, now let’s do it practically what we have learned so far. In the above repository i have give you the Dockerfile for frontend and the backend.

And you also have the docker-compose.yml file in this repo.

So now you just have to do these steps in order to spin up the frontend and the backend containers simultaneously.

version: '3.8'

services:
  frontend:                                    # This is frontend service
    build: ./front-end-nextjs                  # Build from Dockerfile in this directory
    working_dir: /app                          # Set working directory inside container
    command: sh -c "npm run build && npm start" # Build once container starts, then run app
    ports:
      - "3000:3000"                            # Expose frontend on port 3000
    volumes:
      - ./front-end-nextjs:/app               # Mount local code for dev hot-reload
    environment:
      - NODE_ENV=production                   # Env variable for production setup
    depends_on:
      - backend                               # Ensure backend starts first
    networks:
      - app_network                           # Join the shared network

  backend:                                    # This is backend service
    build: ./backend                          # Build backend service from its Dockerfile
    ports:
      - "8000:8000"                           # Expose backend API on port 8000
    env_file:
      - ./backend/.env                        # Load sensitive config like DB URL, secrets
    volumes:
      - ./backend:/usr/src/app                # Mount backend code for live updates (dev)
    environment:
      - FASTAPI_ENV=production                # Custom variable for backend environment
    networks:
      - app_network                           # Same network to talk with frontend

networks:
  app_network:
    driver: bridge                            # Default bridge network for isolated common

To build and start the services in our above compose file:

docker compose up

CONTAINER ID   IMAGE                    COMMAND                  CREATED          STATUS          PORTS                    NAMES
0a771b5eaf4c   devops-url2qr-frontend   "docker-entrypoint.s…"   21 minutes ago   Up 21 minutes   0.0.0.0:3002->3000/tcp   devops-url2qr-frontend-1
d379bafbbd41   devops-url2qr-backend    "uvicorn main:app --…"   23 minutes ago   Up 21 minutes   0.0.0.0:8000->8000/tcp   devops-url2qr-backend-1

And after successful build and run the service you can access them, for this application we are running you can find the:

Fronetend Running on:

http://localhost:3000

Backend Running on:

http://localhost:8000

DockerHub:

Docker Hub is a cloud-based repository where you can store, share, and manage Docker container images.

Official Website

This is the workflow if you want to push your docker image to DockerHub:

# Login to Docker Hub
docker login

# Tag your image
docker tag myapp:latest username/myapp:1.0

# Push it to Docker Hub
docker push username/myapp:1.0

# Pull it from Docker Hub (any machine)
docker pull username/myapp:1.0

If you're working in production or need tighter access control, you might consider private registries like:

• AWS Elastic Container Registry (ECR),

• Azure Container Registry (ACR)

• Google Container Registry (GCR)

• GitHub Container Registry

• GitLab Container Registry

• Harbor

Tools Used

Let’s understand the architecture diagram

1. Source (GitHub Repository)

   • The pipeline starts when a code change is pushed (commit) to your GitHub repository.

   • This event triggers the pipeline automatically — no manual deployment needed.

2. Build Phase (CodePipeline + CodeBuild)

   • AWS CodePipeline detects the commit and initiates the CI/CD process.

   • It hands over the code to AWS CodeBuild, which:

     • Builds the project (compiles the Spring Boot application).

     • Runs tests (unit/integration).

     • Returns the build status (success or failure) back to CodePipeline.

3. Deploy Phase (Elastic Beanstalk)

   • If the build is successful, CodePipeline proceeds to the Deploy stage.

   • It deploys the built Spring Boot application to AWS Elastic Beanstalk, a managed environment that handles infrastructure, load balancing, scaling, and app hosting.

Step-by-Step Follow-up:

Step 1: Repository Setup

First, create a GitHub repository for your Spring Boot project. This will serve as our source code repository.

GitHub Repository: https://github.com/Suraj-kumar00/aws-springboot-ecommerce

Step 2: Elastic Beanstalk Configuration

Create your Elastic Beanstalk environment with these configurations:

• Give it the Application name And the Domain Name and also check the availability.

• Select Java 17 as the platform

• Upload your initial .jar file and set version to 1

• Configure service access settings:

  • Set up service role

  • Configure EC2 key pair

  • Set up AWS Elastic Beanstalk profile

• In initial case there’s no EC2 instance profile so just click on the view permission details and create the role according to that in AWS IAM Role .

Step 3: Database Configuration

Choose the:

• VPC ( In my case I’m choosing default)

• Choose Instance subnets

• Choose Database subnets

For the database setup:

• Enable the RDS database integration

• Configure database username

• Set secure database password

After that just click on next.

• Now in this step just choose the Security Group and we can edit it lated in the EC2 security group inbound rule for opening the custom ports:

Other then that leave all the settings defalut and go to the next step.

• Now choose the system as Basic:

• And leave all the setting as it is but change the ENVIRONMENT VARIABLES :

After that click on the next step.

At the last step just review all the configuration and submit it.

Step 4: Pipeline Setup

Create your AWS CodePipeline:

• Navigate to AWS CodePipeline and create a new pipeline

• Now chose the Custom build pipeline

• Now give the pipeline a name and leave all the setting default and click on the next.

• Now Select the provider which GitHub in my case using OAuth.

  After Authenticating chose the Repository name and the branch and click on next.

• Now select the Other build providers and create a new project:

Give the project name:

After that chose the buildspec.yml options and click on the create code pipeline.

• After that choose it will redirect you to the same page and just click on the next step:

• Now the in the test stage choose the provder AWS CodeBuild and the project name and continute:

• Now in the deploy stage choose the provider as AWS Elastic Beanstalk ,

  Region, Application Name and the Environment Name.

Essential Configuration Files

Add this buildspec.yml to your project root:

You should change the .jar because you name could be different.

version: 0.2
phases:
  install:
    runtime-versions:
      java: corretto17
  pre_build:
    commands:
      - echo Build started on `date`
  build:
    commands:
      - mvn clean install
artifacts:
  files:
    - target/Shopping_Cart-0.0.1-SNAPSHOT.jar
    - Procfile
    - .ebextensions/**/*
  discard-paths: no

5. Pipeline Execution Flow Diagram

This is the execution result of the AWS Code Pipeline Taking the source, Building it and Deploying the new verion of the application to AWS Elastic Beanstalk .

6. Result:

The Application was deploy on this URL: http://aws-springboot-ecommerce.ap-south-1.elasticbeanstalk.com/ (This will not work now as I deleted the steup)

7. Common Challenges and Solutions

Database Connection Issues:

• Problem: Build failures due to communication link issues

• Solution: Implement H2 in-memory database for testing

JAR File Naming:

• Problem: Deployment failures due to JAR file mismatches

• Solution: Maintain consistent artifactId naming across configurations

Best Practices

Key practices to follow:

• Separate test and production configurations

• Use environment variables for sensitive data

• Implement proper proxy configuration

• Add post-deployment hooks

• Maintain consistent artifact naming

• Implement proper error handling and logging

Thanks for reading - See you in the next one!

This guide outlines the process of deploying a SpringBoot application on AWS EC2 with GitHub Actions for automated deployment. The setup includes Docker containers and proper security configurations.

Prerequisites

• AWS Account

• GitHub Repository

• Basic knowledge of Docker and AWS services

• SpringBoot application ready for deployment

NOTE: While working on this project I have made the repository private, so remember

• If your repo is private, then you must configure SSH or PAT on EC2 to interact with it.

• If your repo is public, no key setup is needed to clone it.

But for your reference if you want the application I’m making it public

Here is the Repository

https://github.com/Suraj-kumar00/scm-springboot-application-devops

First let’s understand the architecture Diagram

Architecture Overview

1. User accesses the app via a web browser using the HTTP.

2. The request hits Nginx running on the EC2 instance, which listens on port 80.

3. Nginx acts as a reverse proxy and forwards the request to the Spring Boot application running inside a Docker container on port 8081.

4. The Spring Boot app processes the request and, if needed, communicates with the MySQL database (also running in a Docker container on the same EC2 instance).

5. The response is sent back through the same path:
   MySQL → Spring Boot → Nginx → User's browser.

6. GitHub Actions is used to automatically deploy updates to the EC2 instance by SSHing in and running the necessary Docker commands (e.g., docker-compose up).

Step 1: Launch EC2 Instance

Begin by setting up your EC2 instance with these specifications:

• Choose Ubuntu as the operating system

• Select t2.medium instance type

• Create and download a new key pair for SSH access

• Configure security group with the following ports:

• TCP 22 (SSH)

• TCP 80 (HTTP for Nginx)

• TCP 443 (HTTPS)

• TCP 8081 (Spring Boot via Docker)

• TCP 3000 (phpMyAdmin)

• Set EBS volume size (20GB recommended for free tier)

Step 2: Configure Elastic IP

1. Navigate to Elastic IP section in AWS Console

2. Allocate new Elastic IP address

3. Associate it with your EC2 instance

4. Note down the Elastic IP for future use

Step 3: SSH Into EC2 Instance

ssh -i "your-key.pem" ubuntu@your-elastic-ip

Step 4: Install Docker and Docker Compose

Create and execute this installation script:

#!/bin/bash

# Install Docker
sudo apt update
sudo apt install docker.io -y
sudo systemctl enable docker
sudo usermod -aG docker $USER

# Install Docker Compose
sudo curl -L "<https://github.com/docker/compose/releases/latest/download/docker-compose-$>(uname -s)-$(uname -m)" \\
  -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Step 5: Set Up SSH for GitHub

1. Generate SSH key on EC2:

ssh-keygen -t rsa -b 4096 -C "ec2-rsa-key"

1. Add the public key to GitHub:

• Copy the content of ~/.ssh/id_rsa.pub

• Add it to GitHub under Settings > SSH and GPG Keys

Step 6: Clone and Deploy Application

git clone git@github.com:your-username/your-repo.git
cd your-repo
docker-compose up --build -d

Step 7: Configure Nginx as Reverse Proxy

Install and configure Nginx:

sudo apt install nginx -y
sudo nano /etc/nginx/sites-available/default

Add this configuration:

server {
    listen 80;
    server_name your-elastic-ip;

    location / {
        proxy_pass <http://localhost:8081>;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Restart Nginx:

sudo systemctl restart nginx

Step 8: Set Up GitHub Actions

Create .github/workflows/deploy.yml in your repository:

name: Deploy to EC2

on:
  push:
    branches:
      - main
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Setup SSH Key
        run: |
          echo "${{ secrets.EC2_SSH_PRIVATE_KEY }}" > key.pem
          chmod 600 key.pem

      - name: Deploy via SSH
        run: |
          ssh -o StrictHostKeyChecking=no -i key.pem ${{ secrets.EC2_USER }}@${{ secrets.EC2_HOST }} << 'EOF'
            cd /home/ubuntu/your-repo
            git pull origin main
            docker-compose down
            docker-compose build --no-cache
            docker-compose up -d
          EOF

Step 9: Configure Auto-restart on Reboot

Set up a cron job:

crontab -e
# Add this line:
@reboot cd /home/ubuntu/your-repo && docker-compose up --build -d

Result!!

Common Challenges and Solutions

• Permission Issues

  • Docker permission denied: Run sudo usermod -aG docker $USER

  • SSH key issues: Verify proper key permissions (chmod 600)

• Networking Issues

  • 502 Bad Gateway: Check if Spring Boot container is running

  • Connection refused: Verify security group settings

Best Practices

• Always use environment variables for sensitive data

• Regularly backup your application data

• Monitor application logs and performance

• Keep Docker images updated with security patches

• Use proper version tagging for Docker images

Thanks for reading - see you in the next one!

YAML is a human-friendly data serialization language that's used across all programming languages.

• It’s not a programming language, it’s a data format.

• It’s similar to JSON and XML.

• You can only store data, not run commands.

YAML FULL FORM:

• Previously: "Yet Another Markup Language"

• Now: "YAML Ain't Markup Language"

But Why This Change?

Markup languages like HTML are designed to store documents. But YAML? It can do more than just that — it can store structured data, like objects. That’s why it’s called “YAML Ain’t Markup Language” now — to clarify that it’s not limited to markup.

Data Serialization:

• Serialization is the process of converting a data object—a combination of code and data represented within a region of data storage—into a series of bytes that saves the state of the object in an easily transmittable form. In this serialized form, the data can be delivered to another data store (such as an in-memory computing platform), application, or some other destination.

• In simple words, data serialization is the process of converting an object into a stream of bytes to save or transmit it more easily.

• The reverse of this process is called De-Serialization

Data Serialization languages:

• YAML

• JSON

• XML

Where YAML Is Used?

• Kubernetes config files (*.yaml)

• Docker Compose

• GitHub Actions workflows

• CI/CD pipelines

• Logs, caching systems, etc.

Benefits of YAML:

• Simple & easy to read

• Clean syntax (but indentation matters!)

• Converts easily to JSON, XML

• Language-agnostic (used in most stacks)

• Can represent complex data structures

• Parsing YAML is easy using various tools/parsers

Creating a YAML File

YAML uses key-value pairs and lists. Think of it like dictionaries in Python or HashMaps in Java.

Also:

• YAML is case-sensitive

• Indentation is mandatory

Strings and Multiline Texts

myself: Suraj kumar
fruit: "Apple"
job: 'DevOps'

bio: |
  Hey there my name is Suraj and currently learning YAML
  I am a very enthusiastic about learning new things.

message: >
  this will
  all be 
  in one single line

# Same as:
message: !!str this will all be in one single line

| → preserves newlines
> → converts into a single line (removes newlines)

Numbers and Booleans

In YAML, you can represent all types of numbers — integers, floats, binary, hex, exponential — and even booleans with chill syntax.

number: 6549864          # Integer
marks: 45.12             # Float

booleanValue: !!bool No  # Boolean → false
# Valid false: no, n, false, False
# Valid true: yes, y, true, True

zero: !!int 0
positiveNum: !!int 55
negativeNum: !!int -55
binaryNum: !!int 0b11001     # Binary
octalNum: !!int 0123         # Octal
hexa: !!int 0x45             # Hexadecimal
commaValue: !!int +540_000  # 540000 (underscores allowed)

exponential_numbers: 5.125E56 # Exponential float

marks: !!float 57.98
infinite: !!float .inf        # Infinity
not_a_num: .nan               # Not a number

🧠 YAML even understands underscores in large numbers and scientific notation!

Null Values

YAML supports nulls (empty/missing values) using null, Null, ~, or just leaving it blank.

surname: !!null Null  # null key with null value
~: This is a null key  # Null key with a value

🔥 You can use nulls to mark something intentionally empty.

Dates and Time

Dates and timestamps are built-in. You can write simple dates or full timestamps (with or without timezone).

date: !!timestamp 2024-2-4
India time: 2024-2-4T02:59:43.10 +5:30
no_time_zone: 2024-2-4T02:59:43.10

✅ YAML can auto-parse this as date/time objects in most languages.

Sequence Datatype (Lists)

Lists in YAML are written with dashes - or inline with square brackets [].

student: !!seq
 - marks
 - name 
 - roll_no

# Same as:
student: [marks, name, roll_no]

# Sparse list → Some values can be left empty
sparse_seq: 
 - hey
 - how 
 - 
 - Null
 - sup

🚀 Great for defining arrays or ordered values.

Nested Sequences

YAML also supports nested lists (lists inside lists):

- 
 - mango 
 - apple
 - banana
-
 - marks
 - roll_no
 - date

It’s like having arrays inside arrays 📦

Mapping (Key:Value) or Dictionary

Basic dictionary in YAML is just key: value. You can also nest them or use inline format.

name: Suraj kumar
role: 
   age: 21
   job: student

# Inline version:
role: {age: 21, job: student}

💡 Use this for configs, JSON-style data, etc.

!!pairs (Duplicate keys allowed)

When you wanna allow duplicate keys, use !!pairs.

pair_example: !!pairs
 - job: student
 - job: teacher

# Same as:
pair_example: !!pairs [job: student, job: teacher]

🧩 This is useful for APIs that accept repeated fields.

!!set (Unique values only)

If you want a set of unique keys, use !!set:

names: !!set
 ? suraj
 ? mohit
 ? mahesh

Each key in a set is unique and holds a null/empty value by default.

!!omap (Ordered dictionary)

When the order of key-value pairs matters, go for !!omap:

people: !!omap
 - Suraj:
     name: Suraj kumar
     age: 21
     height: 678
 - Mohit:
     name: Mohit joping
     age: 22
     height: 768

🎯 Super helpful when ordering is important (like user rankings, configs, etc.)

Reusing Properties with Anchors (& and *)

You can reuse data using anchors (&) and aliases (*):

liking: &likes
  fav_fruit: apple
  dislikes: grapes

person1:
  name: Suraj kumar
  <<: *likes

person2:
  name: Mohit joping
  <<: *likes
  dislikes: berries

So basically:

• &likes defines a reusable block.

• *likes pulls that block wherever you want.

• <<: *likes merges it into another object.

🙌 This avoids repetition in large configs.

Storing Data in XML (Extensible Markup Language)

XML is a markup language used to store and transport data. It looks like HTML but is designed for data, not UI.

<?XML version="1.0" encoding="UTF-8"?>
<College name="GITM" principal="Someone">
    <Students>
        <Student>
            <rollno>9</rollno>
            <name>"Suraj"</name>
            <marks>80</marks>
        </Student>       
    </Students>
</College>

Explanation:

• <?XML ... ?> → This is the declaration. It tells the parser this is an XML doc.

• <College> is the root element. It has attributes like name and principal.

• Inside, we nest <Students>, and inside that, a <Student> with info.

• XML is strict with opening/closing tags and proper nesting.

✅ Used in many legacy systems, APIs, config files, and document storage.

Storing Data in JSON (JavaScript Object Notation)

JSON is the go-to format for modern APIs and apps — lightweight, easy to read/write, and works across all programming languages.

{  
   "College": [ 
       {
          "name": "GITM",
          "principle": "Someone",
          "Students": [
                 {
                    "rollno": 2,
                    "name": "Suraj kuamr",
                    "marks": 78
                  }
            ]
        }
     ]
}

Explanation:

• JSON uses {} for objects (like dictionaries) and [] for arrays (lists).

• Everything is in key: value pairs.

• Strings must be in double quotes (" ").

• Commas are required between fields (unlike YAML).

🔥 JSON is everywhere — web apps, APIs, config files, databases, you name it.

YAML DevOps tools

Blog:How to validate Kubernetes YAML files

Here are some tools used to validate and manage YAML configs in the DevOps world:

1. Datree

Used to validate Kubernetes YAML files and catch misconfigurations.

2. Monokle Desktop IDE

A dedicated IDE for managing Kubernetes YAMLs with preview and validation.

3. Lens IDE

Kubernetes IDE to manage clusters and YAML configs visually.

This YAML learning I documented from kunal kushwaha's DevOps course. This is easy isn't it. So this is for today I'll be come with the next blog of DevOps practices soon.

Let's say a developer is working on a project and has set up all the required dependencies and configurations. Everything works fine. However, when a team collaborates on the same project across different operating systems, they may have different dependencies and setups, making it a hassle to configure the environment repeatedly.

For example, if a tool is built for Windows, it won’t run on macOS or Linux, making setup even more difficult. Managing multiple environments and ensuring consistency becomes a challenge.

Docker solves this problem by creating a standardized, portable environment that works the same across all systems, eliminating compatibility issues and simplifying setup.

Before Docker:

• Each application required a dedicated server.

• Running multiple applications on the same server was difficult due to conflicts between dependencies and configurations.

• Companies had to invest in more servers and powerful CPUs, leading to high costs and inefficient resource utilization.

• This approach was not scalable, cost-effective, or environmentally friendly.

Who Solved This Problem?

• Virtual Machines (VMs) provided a solution by allowing multiple applications to run on the same physical server.

• This was an improvement but still had limitations.

Problems with Virtual Machines:

• Each VM required its own full operating system, consuming significant resources.

• Managing dependencies across different environments was still complex.

• “Works on my machine” issues persisted, making development and deployment inconsistent.

Then Containers Came into the Picture

• Containers allow applications to run without needing multiple OS installations.

• Docker simplified containerization, making it easier to build, share, and deploy applications.

• Containers are lightweight, fast, and portable across environments.

• Better resource utilization, run more apps on the same hardware.

• Quick to build, destroy, and deploy, ideal for cloud scaling.

• Standardized workflow from development to production.

Difference between Virtual Machine & Containers

1. Virtual Machines (VMs)

• Each VM runs on a Hypervisor, which allows multiple VMs to share the same physical infrastructure.

• Every VM has its own Guest OS, making it heavyweight since each OS requires its own resources (RAM, CPU, Storage).

• Applications inside VMs come with their own dependencies (Bins/Libs), which can lead to duplication of resources.

• Overhead: Since each VM has a separate OS, it consumes more CPU and memory, making it less efficient for running multiple applications.

2. Containers

• Containers share the same Host OS but are managed by a Container Engine (e.g., Docker, CRI-O, etc.), eliminating the need for separate Guest OS instances.

• Each container has its own dependencies (Bins/Libs) but shares the OS kernel with other containers, making them lightweight and efficient.

• Faster startup and lower overhead compared to VMs since containers don’t require a full OS boot.

• Ideal for microservices and scalable applications since they consume fewer resources and can be deployed rapidly.

Key differences in short:

What is Docker?

Docker is a containerization platform that simplifies the process of building, shipping, and running applications in isolated environments called containers. Under the hood, Docker utilizes Linux namespaces and cgroups to create lightweight, portable containers. Initially, Docker used LXC (Linux Containers), but later switched to its own container runtime called containerd

According to Docker’s official site:

Docker is an open platform for developing, shipping, and running applications. It allows developers to package software into standardized units called containers, which include everything needed to run: libraries, system tools, code, and runtime.

Why Docker?

• Manages Containers Easily – It provides tooling to handle containers efficiently.

• Simplifies Deployment – Packages an app and its dependencies into a single unit (Docker container).

• Build Once, Run Anywhere – Ensures consistency across development, testing, and production.

• Fast & Lightweight – Containers are quick to create, destroy, and scale.

Installation of Docker

I have listed the official website to install the Docker Engine and Docker Desktop for up to date installation steps.

Official Documentation for Installation(Check according to your device)

Website: Install Docker Desktop

Website: Install Docker Engine

Docker Daemon:

The Docker Daemon (dockerd) is the core engine that runs in the background. It listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. The daemon handles all the container operations like starting, stopping, and scaling containers, as well as pulling and creating images.

Docker Desktop:

Docker Desktop is an application for MacOS and Windows that provides a GUI for managing Docker environments. It includes Docker Engine, Docker CLI client, Docker Compose, Kubernetes, and a graphical user interface that allows users to manage containers, images, and other Docker resources visually. Docker Desktop simplifies the process of setting up and using Docker on your local machine.

How Docker Works?

Docker follows a simple process to create, distribute, and run containers. Here’s how it works step by step:

Docker Architecture

1. Docker Client

The developer interacts with Docker using commands like:

• docker build → Creates a Docker Image from a Dockerfile.

• docker pull → Downloads an existing image from the Docker Registry (like Docker Hub).

• docker run → Creates and starts a container from an image.

2. Docker Daemon

• The Docker Daemon runs in the background and manages everything, including images, containers, networks, and storage.

• It processes the client commands and does the actual work of building, pulling, and running containers.

3. Docker Host

• The Docker Host is where containers run.

• It holds two key components:

  • Images: Pre-built packages that contain applications and dependencies (e.g., Ubuntu, Nginx).

  • Containers: Running instances of Docker images that execute applications.

4. Docker Registry

• This is a storage location for Docker images like you store you code on GitHub.

• Common registries include Docker Hub, Google Artifact Registry,Azure container registry, Amazon Elastic Container Registry (ECR), GitHub Conainer registry(GCR) and private registries.

• Images can be pulled (docker pull) from here or pushed (docker push) to share with others.

Process Flow

Step 1: You build an image using docker build .

Step 2: If the image isn’t available locally, Docker pulls it from the registry using docker pull.

Step 3: You run a container using docker run, which creates an instance from the image.

What is a Dockerfile?

A Dockerfile is a script containing a set of instructions to build a Docker Image. It defines the base OS, dependencies, and commands required to set up an environment.

Docker Image

A Docker Image is a lightweight, standalone, and executable software package that includes everything needed to run an application: code, runtime, system tools, libraries, and dependencies. A docker image is built in layers, where each instruction in the Dockerfile creates a new layer. These layers stack on top of each other to form the final image.

1. FROM → Base image layer (e.g., Ubuntu, Node.js, or Alpine).

2. WORKDIR → Sets the working directory inside the container.

3. COPY → Copies files from the host to the container.

4. RUN → Executes commands inside the image (e.g., installing dependencies).

5. EXPOSE → Declares which ports the container will use.

6. CMD / ENTRYPOINT → Defines the default command when the container starts.

Each layer is immutable—once created, it doesn’t change. Instead, new layers are added when modifications are made.

• Images are immutable (cannot be changed).

• Stored locally or on Docker Hub for sharing.

Writing your first Dockerfile:

FROM ubuntu:latest  

WORKDIR /app  

COPY package.json package-lock.json ./  

COPY . .  

EXPOSE 3000  

CMD ["npm", "start"]

Docker Layer Caching

Docker optimizes image builds by caching layers. If a layer hasn’t changed, Docker reuses the cached version instead of rebuilding it.

How caching works:

• Docker runs through the Dockerfile line by line.

• If a layer hasn’t changed, it is pulled from cache.

• If a layer changes, all layers after it must be rebuilt.

Containers

A Docker Container is a running instance of an image. It provides a lightweight and isolated environment to run applications.

• Ephemeral (data is lost when deleted unless using volumes).

• Multiple containers can be created from the same image.

Recap with Demo: Dockerizing a Nodejs application

Remember above we have written our first Dockerfile we will be using that.

Now for the practice at your own you can use this github repository, here you will find the nodejs applicaiton code okay: Nodejs Application Code

Dockerfile

Step 1: Writing the Dockerfile

# Use the latest Ubuntu image as the base
FROM ubuntu:latest

# Set the working directory inside the container
WORKDIR /app

# Copy package files first (helps with caching layers efficiently)
COPY package.json package-lock.json ./

# Install dependencies
RUN apt update && apt install -y nodejs npm && npm install

# Copy all remaining project files
COPY . .

# Expose port 3000 to allow external access to the application
EXPOSE 3000

# Start the application using npm
CMD ["npm", "start"]

Step 2: Build the Docker Image

Now, let's build the image using the Dockerfile:

docker build -t mynodeapp .

• t mynodeapp → Gives the name mynodeapp to the image.

• . → Refers to the current directory where the Dockerfile is located.

Step 3: Run the Container

Once the image is built, we can start a container from it:

docker run -d -p 3000:3000 --name mynodejscontainer mynodeapp

• d → Runs the container in detached mode (in the background).

• p 3000:3000 → Maps port 3000 of the container to port 3000 of the host.

• -name mynodejscontainer → Names the running container mynodejscontainer.

• mynodeapp → Uses the mynodeapp image to create the container.

Step 4: Verify the Running Container

To check if the container is running:

docker ps

CONTAINER ID   IMAGE       COMMAND        STATUS        PORTS                  NAMES
abcdef123456   mynodeapp   "npm start"    Up 5 minutes  0.0.0.0:3000->3000/tcp  mynodejscontainer

Step 5: View Logs of the Application

To check logs and ensure the app is running properly:

docker logs -f mycontainer

> mynodeapp@1.0.0 start /
> node server.js
Server is running on port 3000...

Step 6: Access the Application

Now, open your browser and go to:

http://localhost:3000

Now, we have successfully dockerized our Node.js application running inside a Docker container!

This is the flow form writing a dockerfile, building the image out of it and running the container of the image.

Basic Docker Commands

Run a container interactively

# Runs an Ubuntu container in interactive mode (-it)
docker run -it ubuntu

# Run the container using port you have exposed in dockerfile, for example -p (publish)
docker run -it -p hostPort:containerPort <name-of-the-image>

List all running containers

# Shows currently running container
docker container ls

# or 
docker ps

List all containers (including stopped ones)

# Lists all containers, including stopped ones
docker container ls -a

Stop a running container

# Stops a running container using its container ID
docker container stop [container_id]

Remove all stopped containers, unused volumes, and networks

# Cleans up unused containers, networks, and images
docker system prune

Exec into the running container

docker exec -it <name-of-your-container> bash

Inspect running processes in a container

# Lists all running processes inside the container after you exec into the container
ps -ef

Image & Container Management

List all Docker images

# Shows all locally available Docker images
docker images

Pull an image from Docker Hub

# Downloads the latest Nginx image
docker pull nginx

Remove an image

# Deletes the specified Docker image
docker rmi nginx

Start a stopped container

# Restarts a stopped container
docker start [container_id]

# Or you can use the name of the container as well
docker start <name-of-the-conatiner>

Restart a container

# Stops and starts a container again
docker restart [container_id]

# Or you can use the name of the container as well
docker restart <name-of-the-conatiner>

Delete a container

# Removes a stopped container permanently
docker rm [container_id]

# Or you can use the name of the container as well
docker rm <name-of-the-conatiner>

Container Logs & Inspection

View container logs

# Shows logs of a specific container
docker logs [container_id]

# Or you can use the name of the container as well
docker logs <name-of-the-conatiner>

Check container details

# Displays detailed information about a container
docker inspect [container_id]

# Or you can use the name of the container as well
docker inspect <name-of-the-conatiner>

Monitor real-time container stats

# Shows real-time CPU, memory, and network stats of containers
docker stats

I’ll be covering these advanced and Important topics in next blog

1. Docker Compose

2. Docker Networking

3. Docker Volumes

4. Docker Swarm

5. Docker Vs Kubernetes

The End!

I hope you enjoyed this blog and have learned about docker today. See you in the next one.

The Problem It Solves

Anyone can host a static website on AWS, but that’s not enough in today's landscape. The real challenge is building a secure, fast, and scalable website that efficiently uses AWS resources.

• Security Risks: Hackers are everywhere, and a simple HTML file won’t stop them.

• Performance Issues: Users expect lightning-fast websites. Anything over a second and they bounce.

• Cost Management: Resources need to be allocated smartly to avoid unnecessary spending.

This blog walks you through building a highly secure, performance-optimized static website leveraging AWS services.

How I Built It

The AWS services used:

• AWS S3: Foundation for static site storage.

• CloudFront: Distributes content worldwide for ultra-fast loading.

• Certificate Manager: Manages HTTPS encryption.

• AWS WAF: Protects against cyber threats like DDoS, XSS, and SQL Injection.

• CloudWatch: Monitors traffic, security, and performance with alerts.

• AWS Route 53: Manages the custom domain.

This is the architecture for Hosting a Static Website

Understanding the Architecture: The Complete Flow

1️⃣ User Request (Entry Point)

• A user visits https://mindfulcloud.devsuraj.me.

• The request is sent to Amazon Route 53, which maps the domain to CloudFront.

2️⃣ Route 53 → CloudFront

• Route 53 forwards the request to Amazon CloudFront (CDN).

• CloudFront serves cached content to it’s edge locations near to the users for lower latency.

3️⃣ Security Layer

• AWS WAF protects against:

  • SQL Injection

  • Cross-site Scripting (XSS)

  • DDoS attacks

• AWS ACM (AWS Certificate Manager) enables SSL/TLS encryption.

4️⃣ Fetching Content from S3

• If CloudFront doesn’t have the content cached, it fetches it from Amazon S3.

• S3 acts as the origin server for static assets like HTML, CSS, JavaScript, and images.

5️⃣ Monitoring & Logging

• AWS CloudWatch tracks:

  • CloudFront request logs, cache hit ratio, and performance metrics.

  • S3 storage access patterns.

• Alerts notify about security threats or traffic spikes.

Implementation

Step 1: Setting Up an S3 Bucket for Static Website Hosting

1. Go to AWS Console S3 service and Create Bucket.

   

2. Enter a unique bucket name (e.g., mindfulcloud.devsuraj.me).

   

3. Choose public or private access (public if hosting static content).

   

4. Enable static website hosting under Properties.

   

   Scroll down below and enter the Index document and save it.

   

5. Upload your static website files (HTML, CSS, JS).

   

Step 2: Configuring AWS ACM (SSL Certificate) for HTTPS

1. Go to AWS Certificate Manager (ACM) → Request a Certificate.

   You should create the certificate in the N.Verginia otherwise you’ll not be able to see it when you select the certificate in the cloudfront.

   

2. Choose Public Certificate → Enter domain name (e.g., mindfulcloud.devsuraj.me).

   In my case I have my root domain which devsuraj.me so I want subdomain to attach it and above I have mentioned.

3. Choose DNS validation (faster & recommended).

   

4. ACM provides CNAME records; add them to your Namecheap DNS settings.

   Copy the CNAME name and CNAME value.

   

   Add it to the DNS service provide you have in my case I have Namecheap:

   1. After login to you namecheap to domain list and click on manage and go to the advanced DNS.

      

      

   2. Now create new record

      

   3. Enter the CNAME and The Value

      

5. Now you’re done with ACM, wait for validation (may take a few minutes to hours).

Step 3: Setting Up a Route 53 Hosted Zone

1. Go to AWS Route 53 → Create Hosted Zone.

   

2. Enter your subdomain (e.g., mindfulcloud.devsuraj.me).

   

3. Copy the provided NS (Name Server) records and update them in Namecheap’s DNS settings.

   

   

4. Create an A record (alias) pointing to CloudFront Distribution.

   

Step 4: Setting Up CloudFront for CDN & Caching

1. Go to AWS CloudFront → Create Distribution.

   

2. Set Origin Domain as your S3 bucket.

   

3. Enable HTTPS using your ACM certificate.

   

   Select you ACM certificate:

   

4. Configure default root object (index.html).

   

5. Deploy the distribution and note the CloudFront URL.

6. After creating the distribution it you’ll give you the bucket policy to you just copy that and paste it in the bucket you created:

   

Step 5: Connecting Namecheap Domain to AWS

1️⃣ Go to Namecheap DNS Settings

• Log in to Namecheap → Domain List → Manage your domain → Advanced DNS.

2️⃣ Verify NS Records are correct( as we did this in step 3)

• Ensure Nameservers are set to Custom DNS with Route 53 NS records.

3️⃣ Save & Wait for Propagation

• Click Save Changes → Wait a few minutes to 24 hours.

• Open https://yourdomain.com in a browser to confirm it works.

Step 6: Enhancing Security with AWS WAF

1. Go to AWS WAF → Create WebACL.

   

2. Attach it to CloudFront Distribution.

   

3. Add security rules:

   • Block common threats (DDoS, SQL Injection, XSS attacks).

   • Rate limiting for excessive requests.

   • Geo-restriction to block traffic from unwanted locations.

4. Enable AWS WAF logging to monitor security threats.

   

Step 7: Enabling CloudWatch Monitoring & Logging

1. Enable CloudFront Logging:

   • Go to CloudFront → Enable Standard Logging.

   • Choose an S3 bucket to store logs.

2. Create CloudWatch Dashboards:

   • Monitor requests, latency, cache hit ratio.

   • Set up CloudWatch alarms for security events.

3. Set Up Alerts:

   • Use AWS SNS (Simple Notification Service) to receive alerts via email/SMS.

   • Example: Alert if CloudFront request count exceeds a threshold.

Select The SNS topic or create one:

4. And just complete the step 3 and step 4 in creating the alarm and you’re good go.

Step 8: Implementing Reliability & Disaster Recovery

1. Enable S3 Versioning:

   • Go to S3 → Properties → Enable Versioning.

   • This helps rollback in case of accidental file deletion.

2. Enable Cross-Region Replication:

   • Go to S3 → Replication Rules → Set up a backup region.

   • Choose a different AWS region for redundancy.

Send the requests to your website to check the performance, WAF security and other metrics.

#!/bin/bash

URL="your website url"
REQUESTS='' # include the number of requrests you want to send  

echo "Starting load test on $URL with $REQUESTS requests..."

for ((i=1; i<=REQUESTS; i++))
do
  curl -s -o /dev/null -w "Request $i: HTTP %{http_code}\n" $URL &
done

echo "Load test initiated!"

How to Run

1. Save this script as simple_load_test.sh.

2. Give it execution permission:

chmod +x simple_load_test.sh

3. Run it:

./simple_load_test.sh

The end!!!

Thanks for the reading guys, see you in the next one…

Amazon Web Services (AWS) is a cloud computing platform that offers a wide range of services such as computing power, storage, databases, and networking. If you're a beginner I’m here to help you with my experience of learning and building project in AWS infrastructure, AWS provides scalable and flexible solutions for building applications.

In this guide, we'll be exploring the fundamentals of AWS and walk through the setup of Identity and Access Management (IAM), which is essential for securing your AWS environment.

What is AWS?

AWS is a cloud computing service provided by Amazon that offers on-demand computing resources over the internet with pay-as-you-go pricing. It eliminates the need for physical hardware, reducing costs and increasing efficiency.

Key Features of AWS:

✅ Scalability – Automatically scales based on demand.

✅ Pay-as-you-go Pricing – No upfront costs; pay only for what you use.

✅ Security & Compliance – Built-in security measures and compliance with industry standards.

✅ Global Infrastructure – Data centers across multiple regions for high availability.

✅ Wide Range of Services – Compute, storage, databases, AI/ML, and more.

Popular AWS Services:

• EC2 – Virtual machines for computing power.

• S3 – Secure object storage.

• RDS – Managed relational database service.

• Lambda – Serverless computing.

• VPC – Private cloud networking.

We’ll discuss most used AWS services in the Cloud/DevOps space in the next blog.

Create your AWS account now!

Prerequisites:

1. You’ll need a valid Email Address

2. A Payment Method (debit card for example) - don’t worry, broh 😂, they’re not here to swipe your cash!

Steps to Create an AWS Account:

My AWS Console:

Understanding IAM (Identity and Access Management)

AWS IAM is a security service that allows you to manage users, permissions, and access to AWS resources.

Why is IAM Important?

🔒 Access Control: Defines who can access AWS resources.

🔒 Least Privilege Principle: Assigns only the required permissions to users.

🔒 Multi-Factor Authentication (MFA): Enhances security with an additional authentication step.

🔒 AWS Root User Protection: Avoid using the root account for daily tasks.

IAM Components:

1. Users: Individual AWS accounts that have credentials (such as a username, password, or access keys). Each user represents a person or an application that interacts with AWS resources.

   

2. Groups: Collections of users that share the same set of permissions. This makes it easier to manage access controls for multiple users at once.

   

3. Roles: Entities that define a set of permissions for AWS services or external identities. Roles allow you to grant temporary access without having to share long-term credentials.

   

4. Policies: JSON-based documents that define permissions. They specify what actions are allowed or denied on which AWS resources. We can create custom policies which is the used according to use-case.

   Example: A policy is a JSON document that specifies allowed or denied actions. For example, the S3ReadOnlyAccess policy grants read-only access to an S3 bucket.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::example-bucket/*"
        }
      ]
    }
   

   

Setting Up IAM in AWS

IAM Dashboard:

Follow these steps to configure IAM for a secure AWS environment.

Step 1: Create an AWS IAM User

1. Log in to AWS Management Console using the root account.

2. Navigate to IAM → Click on Users.

3. Click Add user and specify the user details

   

4. Click Next → set Permissions, Choose the permission options and if you have created group you can attack to it.

   

5. Click Next → Add Tags (Optional).

6. Click Create User → Download credentials (.csv file).

Step 2: Create an IAM Group & Assign Policies

1. Go to IAM → Click on Groups.

2. Click Create Group → Enter Group Name.

3. Attach a predefined IAM policy (e.g., AdministratorAccess, ReadOnlyAccess).

4. Add Users to the group → Click Create Group.

Step 3: Enable MFA for Extra Security

1. Navigate to IAM Users → Select the User.

2. Click Security Credentials → Enable Multi-Factor Authentication (MFA).

   

3. Use an Authenticator App (Google Authenticator/AWS MFA) to scan the QR code.

4. Enter the generated MFA codes → Click Assign MFA.

Step 4: Use IAM Role for AWS Services

An IAM Role is an AWS identity with a set of permissions that determine what actions are allowed or denied. Unlike IAM users, roles do not have long-term credentials (passwords or access keys). Instead, they provide temporary security credentials to AWS services or external identities.

1. Go to IAM → Click Roles → Create Role.

2. Choose AWS Service (e.g., EC2, Lambda).

   

3. Attach a Policy (e.g., AmazonS3FullAccess).

4. Click Create Role.

Best Practices for AWS IAM Security

✅ Use IAM Users Instead of Root Account – The root account should be used only for account setup and billing.

✅ Follow the Principle of Least Privilege – Grant only necessary permissions.

✅ Enable MFA for All Users – Adds an extra layer of security.

✅ Rotate Access Keys Regularly – Prevents unauthorized access.

✅ Monitor IAM Activity with CloudTrail – Track API calls and security events.

✅ Use IAM Roles for Applications – Avoid hardcoding credentials in code.

The end!

Hey guys, thanks for the reading, I hope you have learned something valuable today via this blog. Let me know in the comment if you have any doubt or anything you want to say. I would love to reply.

Imagine renting a fully equipped kitchen instead of building one from scratch. Cloud computing works similarly: instead of owning physical servers or data centers, you "rent" computing power, storage, and services over the internet. This model allows businesses and individuals to focus on innovation without worrying about infrastructure.

Cloud computing is reshaping the world of IT, providing the on-demand delivery of IT resources with pay-as-you-go pricing. This model enables businesses to scale, optimize costs, and access advanced technologies without managing physical infrastructure.

Why This Matters:

• Businesses and developers use the cloud to innovate faster.

• DevOps teams leverage it to automate workflows and deploy apps seamlessly.

Core Characteristics of Cloud Computing

1. Trade Upfront Expense for Variable Expense

Replace large initial investments in hardware with a flexible "pay-as-you-go" model, like a monthly utility bill.

2. Stop Spending Money to Run and Maintain Data Centers

Get rid of the costs and headaches of managing physical servers, cooling systems, and IT teams.

3. Stop Guessing Capacity

Avoid overpaying for unused servers or crashing during traffic spikes—scale resources up or down instantly.

4. Benefit from Massive Economies of Scale

Access cheaper rates for computing power, storage, and services by leveraging the cloud provider’s bulk purchasing power.

5. Increase Speed and Agility

Launch applications in minutes, experiment freely, and adapt to market changes faster.

6. Go Global in Minutes

Deploy your app to users worldwide with a few clicks, no need to build physical data centers.

Types of Cloud Deployments

Cloud computing offers different deployment models based on organizational needs:

• Public Cloud: Services are provided over the internet by third-party providers, offering scalability and cost efficiency.

• Private Cloud: Dedicated infrastructure for a single organization, ensuring greater security and control. Managed in-house or by a provider.

• Hybrid Cloud: Combines public and private clouds, allowing seamless data and application movement for flexibility.

• Community Cloud: Shared infrastructure for organizations with common concerns (e.g., government, healthcare), offering cost-sharing and compliance benefits.

The 3 Core Cloud Service Models

1. IaaS (Infrastructure as a Service)

• What it is: Rent virtual servers, storage, and networks.

• For whom: Developers who want control over infrastructure.

• Analogy: Leasing a plot of land to build your dream house (you handle construction).

2. PaaS (Platform as a Service)

• What it is: Tools to build apps without managing servers.

• For whom: Teams focused on coding, not hardware.

• Analogy: Moving into a furnished apartment (you decorate; the landlord handles plumbing).

3. SaaS (Software as a Service)

• What it is: Ready-to-use apps accessed via the internet.

• For whom: Everyone (e.g., Gmail, Netflix).

• Analogy: Ordering pizza delivery (no cooking, no cleanup).

How Cloud Computing Powers DevOps

DevOps bridges software development (Dev) and IT operations (Ops). The cloud fuels this partnership with:

1. CI/CD Pipelines: Automation Magic

• CI (Continuous Integration): Automatically test code changes.

• CD (Continuous Delivery): Deploy updates to users instantly.

2. Infrastructure as Code (IaC)

• What it is: Define servers and networks using code (e.g., Terraform).

• Why it matters: Spin up identical environments in minutes.

3. Scalability & Elasticity

• Auto-scaling: Apps grow/shrink resources based on traffic.

• DevOps Impact: No more downtime during traffic spikes (e.g., holiday sales).

Debunking Cloud Myths

Myth 1: “The Cloud Isn’t Secure”

Reality: Cloud providers invest more in security than most companies.

Myth 2: “It’s Only for Tech Giants”

Reality: Startups use it to compete with big players (e.g., Dropbox scaled on AWS).

Myth 3: “Cloud = Expensive”

Reality: Turn off unused servers—like shutting lights when you leave a room.

Core Cloud Providers and Their Offerings

1. Amazon Web Services(AWS)

Known for its scalability and vast array of services.

Technical Strengths:

• Service Breadth: Over 200 services, including compute (EC2), storage (S3), databases (RDS), and machine learning (SageMaker).

• Scalability: Auto-scaling groups and Elastic Load Balancing (ELB) handle traffic spikes.

• Global Reach: 32 geographic regions, 102 Availability Zones (AZs) for low-latency deployments.

DevOps Integration:

• CI/CD: AWS CodePipeline for automated workflows.

• Infrastructure as Code: AWS CloudFormation.
  Use Case: Startups and enterprises needing flexibility and global reach.

2. Microsoft Azure

Ideal for businesses using Microsoft tools and products .

Technical Strengths:

• Enterprise Integration: Seamless compatibility with Microsoft products (Active Directory, Office 365, SQL Server).

• Hybrid Cloud: Azure Stack bridges on-premises and cloud environments.

• AI/ML: Azure Machine Learning and Cognitive Services (pre-built APIs for vision, speech).

DevOps Integration:

• CI/CD: Azure DevOps pipelines.

• Monitoring: Azure Monitor and Application Insights.
  Use Case: Enterprises already invested in Microsoft ecosystems.

3. Google Cloud Platform (GCP)

Excellent for analytics and AI services.

Technical Strengths:

• Data & AI: BigQuery (serverless data warehouse), TensorFlow (open-source ML framework).

• Kubernetes: Google Kubernetes Engine (GKE) dominates container orchestration.

• Sustainability: Carbon-neutral operations with commitments to 24/7 renewable energy.

DevOps Integration:

• CI/CD: Google Cloud Build.

• Observability: Cloud Monitoring and Logging.
  Use Case: Data-driven companies prioritizing analytics, AI, or Kubernetes.

Choosing the Right Provider: Key Considerations

1. Ease of Use

2. Cost Structure

3. Service Variety

4. Compliance & Security

5. Support & Ecosystem

Here are the offering of three big cloud provides:

Closing Remarks

Cloud computing isn’t magic, it’s a tool that democratizes technology. Whether you’re a developer automating deployments with DevOps or a small business backing up files, the cloud is your gateway to efficiency and innovation.

In the next blog we’ll discuss about AWS Cloud Platform.

The end - Thanks for reading

Hacktoberfest is an annual month-long celebration of open-source that runs throughout October (Oct 1 - Oct 31).

Why should you participate?

Participating helps you learn new skills, collaborate with others, improve your coding abilities, and give back to the developer community.

Who organizes Hacktoberfest?

Hacktoberfest is organized by DigitalOcean in partnership with GitHub and other companies in the tech community. It has been running annually since 2014.

Official Sponsors and Partners for Hacktoberfest 2025

How Hacktoberfest Works

Rules and guidelines for 2025:

To participate in Hacktoberfest 2025, you must register on the official website at hacktoberfest.com.

Key Requirements:

• Make 6 valid pull/merge requests to public repositories that have the "hacktoberfest" topic added

• Contributions must be to repositories that have opted-in by adding the "hacktoberfest" topic

• Only quality contributions count, and spammy PRs will be disqualified

• Pull requests can be made to any participating repository on GitHub or GitLab

Key dates and timeline:

• Registration opens: Late September 2025

• Event start: October 1, 2025

• Event end: October 31, 2025

• Final review and PR count: PRs are analyzed continuously throughout October

What counts as a valid contribution?

Valid contributions include:

Code Contributions:

• Bug fixes

• New features

• Performance improvements

• Code refactoring

Non-Code Contributions:

• Documentation updates

• Design work

• Translations

• Testing

• Writing tutorials

• Creating educational content

Setting Up for Hacktoberfest (Prerequisites)

1. You should have a GitHub/GitLab account to participate in hacktoberfest.

2. Should be familiar with Git and GitHub
   Git is a version control tool that tracks changes in your code. GitHub is a platform that hosts Git repositories and allows collaboration.

3. Essential tools for contributing

   • Git: Set up Git on your local machine.

   • Code editors: VSCode(recommended) or any other text editor.

Understanding Open-Source

What is open source?

In general, it's free to use, access, and modify depending on the licenses. It's an inclusive community and you get to collaborate with people around the world.

Ways to find projects for Hacktoberfest 2025

1. Good First Issue label:
   good first issue is the best way to start your Open-Source journey. You can find all of these on GitHub by doing a label search. Here is how:

   Link:https://github.com/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22

   

2. GitHub Topics:
   It's a very cool feature on GitHub. With the help of this, you can find a particular project with a topic. The topic can be languages, events, tech, etc.

   Link: https://github.com/topics/javascript

   

3. GitHub's advanced search:

   GitHub advanced search provides tons of filters to find repository and issues according to our preferences. You can filter by language, extension, issue labels, number of stars/forks, etc.

   Link: https://github.com/search/advanced

   

Resource to find project to contribute

1. Awesome-Hacktoberfest

   %[https://github.com/mattjegan/awesome-hacktoberfest]

2. How to contribute to open-source repo of freecodecamp

   %[https://github.com/freeCodeCamp/how-to-contribute-to-open-source]

Step-by-Step Contribution Guide

1. Forking a repository:

   Go to the repository you want to contribute to on GitHub, click the "Fork" button in the top-right corner. This creates a copy of the project under your GitHub account.

   Link: https://github.com/Suraj-kumar00/Getting-started-with-Git-Github

2. Cloning a repository to your local machine

   After forking, clone the repository by using the command:

    git clone <URL of your forked repository>
   

   

3. Making your first change (committing and pushing)

   1. Atiquette of writing a good commit message(good practices):
      Here are common commit message types in short:

      • feat: Adds a new feature.

        • Example: feat: add user login

      • fix: Fixes a bug.

        • Example: fix: resolve login error

      • docs: Documentation changes.

        • Example: docs: update README

      • style: Code style changes (e.g., formatting).

        • Example: style: format code to match guidelines

      • refactor: Code refactoring (no new feature or bug fix).

        • Example: refactor: simplify login logic

      • test: Adds or updates tests.

        • Example: test: add unit tests for auth

      • chore: Routine tasks (e.g., updating build scripts).

        • Example: chore: update dependencies

      • perf: Performance improvements.

        • Example: perf: improve API response time

   2. Key Guidelines for Writing a Good Commit Message:

      1. Use a Clear Structure:

         • Subject line (max 50 chars).

         • Body (optional, explain why).

         • Footer (references issues, if any).

      2. Present Tense:

         • Write as if the change is happening now (e.g., "Fix bug" instead of "Fixed bug").

      3. Be Brief but Specific:

         • Summarize the change concisely (e.g., "Add user authentication" rather than "Update stuff").

      4. Explain Why, Not Just What:

         • Give the reasoning behind the change, not just the action.

      5. Bullet Points for Multiple Changes:

         • If the commit involves several tasks, list them clearly.

      6. Reference Issues/PRs:

         • Mention related issues or pull requests for clarity (e.g., "Closes #123").

      7. Active Voice:

         • Write directly and clearly (e.g., "Add feature" vs. "Feature added").

   3. Example:

       Fix user logout bug
      
       This fixes a bug where users were not fully logged out.
      
       Closes #456
      

4. Opening your first pull request:

   Push the changes to your forked repository using:

    git push origin main
   

5. How to write a good pull request description

   Be clear about what changes you made and why. Link to any related issues. Use a concise and descriptive title for your PR.

Common Mistakes to Avoid

1. Understanding issues and PR guidelines

   Always read the project’s contributing guidelines before submitting PRs. Follow the maintainers' instructions on formatting and coding style.

2. Avoiding low-quality or spammy contributions

   Don’t submit trivial changes just for the sake of completing Hacktoberfest. Only meaningful contributions will count.

3. Properly communicating with maintainers

   When opening an issue or PR, be polite and respectful. Ask for feedback or clarification when necessary.

Navigating Hacktoberfest Challenges

1. Solving merge conflicts

   A merge conflict happens when two people make changes to the same line of code. To resolve it, edit the conflicting file, remove the conflict markers, and commit the changes.

2. Dealing with rejected pull requests

   If your PR is rejected, don’t get discouraged. Review the feedback, make improvements, and resubmit. It’s part of the learning process.

3. How to ask for help (community forums, GitHub discussions, ask to maintainers etc.)

   If you’re stuck, don’t hesitate to ask for help on community forums or GitHub discussions. Be specific about what you need help with.

Completing Hacktoberfest

1. Tracking your progress:

   Once you register for Hacktoberfest, your contributions are automatically tracked on the official website. Make sure your pull requests are accepted before the deadline.

2. Submitting the required pull requests:

   Complete 6 valid pull requests to qualify for the rewards. Contributions across multiple repositories count.

3. What happens after you complete Hacktoberfest? (swag, certificates, etc.):

   Once you successfully complete Hacktoberfest, you will be eligible to claim swag (like a digital badges) or donate a tree. You’ll also receive a digital certificate of completion.

Post-Hacktoberfest: Continuing Your Open Source Journey

1. How to stay involved in open source projects

   Keep contributing to projects that interest you. Hacktoberfest is just the beginning of your open source journey.

2. Finding new projects to contribute to

   Explore GitHub’s Explore tab or join open source communities to find new projects. Most open source projects welcome year-round contributions.

   Link: https://github.com/explore

   

3. Building your portfolio through contributions

   Your GitHub profile is like a resume. Keep working on meaningful projects and build a strong portfolio that showcases your skills.

Join our community:

DevsDeialogue

1. Github: https://github.com/DevsDialogue

2. Discord: https://discord.gg/Q4t66h38

Conclusion

Whether you're an experienced developer or just starting out, Hacktoberfest 2025 is a great way to get involved in open source. You'll learn, grow, and make meaningful contributions to the developer community.

Open source is more than just Hacktoberfest. It's a year-round opportunity, and you can contribute to any open source project at any time based on your interest.

Happy Hacking! 🎃

Thanks for reading and happy contributing to Hacktoberfest 2025!

Python is a programming language which is:

1. Easy to learn

   • simple syntax

   • easy to setup

2. Large ecosystem

   • many libraries

   • large community

3. Flexible

   • you are not limited to language specifics

What Python is used for:

• Web Development

• Data Science

• Machine Learning

• Artificial Intelligence

• Web Scrapping

• Automation

  • Like automating DevOps tasks

  • Automating general tasks

Why Python with DevOps?

As an approach to culture, platform, and automation, Python is an important language for DevOps engineers or teams working to automate repetitive tasks, infrastructure management, provisioning, API driven deployment.

Setup python locally in Ubuntu

Head over to the official Python website and download the latest version of it according to your operating system.

Check that the Python is downloaded to your machine:

# this comamand is for Linux users
python3

# Checking the Python Path
which python3

Python Fundamentals:

Print Function

The print() function in Python is used to output text or other data to the console.

1. objects: Any number of objects to be printed. Multiple objects are separated by a space by default.

2. sep: A string inserted between each object. The default is a space.

3. end: A string appended after the last object. The default is a newline.

4. file: A file-like object (stream). The default is sys.stdout.

# Basic usage
print("Hello, world!")

# Printing multiple items
print("The answer is", 42)

# Changing the separator
print("apple", "banana", "cherry", sep=", ")

# Changing the end character
print("Hello", end=" ")
print("world!")

# Printing to a file
with open("output.txt", "w") as f:
    print("Hello, file!", file=f)

# This is the Output: Hello, file!

# Using formatted strings aka f-strings
name = "Alice"
age = 30
print(f"My name is {name} and I am {age} years old.")

Literals

A literal is a notation for representing a fixed value directly in the source code. Here is an example of a literal:

# Integer Literal:
93
# Floating-point Literal:
3.14
# String Literal:
"Hello, Suraj!"
# Boolean Literal
True
# None Literal: This represents the absesce of a value or null value
None
# Lits Literal:
[1,2,3,4]

# Simple program for literals
integer_literal = 42
string_literal = "Hello"
print(integer_literal, string_literal)

Variable

A variable is a named storage location in memory that holds a value that can be changed during program execution.

Variable Rules in Python

1. Names can contain letters, digits, and underscores: e.g., my_variable1.

2. Names must start with a letter or an underscore: e.g., _variable or variable1.

3. Names cannot start with a digit: e.g., 1variable is invalid.

4. Names are case-sensitive: e.g., Variable and variable are different.

5. Cannot use Python reserved keywords: e.g., class, def, if, etc.

Naming Convention:
Naming conventions is a convention(generally agreed scheme) for naming variables.

• Use lowercase with words separated with Underscores

# Simple program using variables
my_number1 = 10
my_number2 = 20
print("First No.", my_number1, "Second No.", my_number2)

Keywords

Keywords in Python are reserved words that have special meaning and cannot be used as identifiers (variable names, function names, etc.). You don't need to remember all these keywords I just put it over here for reference.

Expressions and statements

Expressions are code segments that evaluate a value, while statements are instructions that perform an action in a program. A program is made up of several statements.

# These are the exmaple of Expressions:

# 1. Arithmetic expression:
2+3
# 2. String concatination
"Hello, "+" World!"
# 3. Function call expression:
len("example")
# 4. Boolean expression:
5>2
# List indexing:
my_list[1]

# Simple program with expressions and statements

a = 5  # this is a statement (assignment statement with an expression on the right side)
b = 10  # this is a statement (assignment statement with an expression on the right side)
result = a + b  # this is a statement (assignment statement with an expression on the right side)
print("Result:", result)  # this is a statement (function call statement with an expression as an argument)

Comments

Comments in Python are used to explain code logic, making it easier for others to understand what the code does. It also helps to document the code’s purpose, usage, and functionality and enhances maintainability by making the code simpler to manage and update.

Comments are of two types:

1. Simple Line Comments that start with '#' and below is an example.

x = 10  # Assigning 10 to x
print(x)  # Printing the value of x

2. Multi-line Comments: that uses triple quotes (''' or """).

"""
This is a multi-line comment.
Hi there suraj this side.
"""

Indentation in Python

Indentation means spacing in statements in Python is very important. For example, in below code snippet below, you can see the print function has a lot more space from its original position which will give an IndentationError: unexpected indent .

# This program will give indentation error
name = "Suraj"
   print(name)

Data Types

Different data types can do different things: We will see two immutable data types in this blog rest of all we'll learn in the journey...

Number Data Types

# Simple program with number data types
pi = 3.14
print("Pi:", pi)

# Output: 3.14

Strings Data Types

# Simple program with strings
message = "Hello, Python!"
print(message)

String Concatenation

Adding strings together.

# String Concatenation program
print("Hello Suraj" + "Are you learning Python?") # we concatenate using '+' symbol

Instead of writing above style, we can usef-string:

# This is a formatted string literal, introduced in Python 3.6
print(f"Hello Suraj, are you {21} years old?")

# Output:
Hello Suraj, are you 21 years old?

String Methods

String methods...

1. upper(): Used to convert all characters in a string to uppercase.

      # Convert to uppercase
      message = "hello, python!"
      print(message.upper())
   
      # Output:
      HELLO, PYTHON!
   

2. lower(): Convert all characters in a string to lowercase.

    # Convert to lowercase
    message = "HELLO, PYTHON!"
    print(message.lower())
   
    # Output:
    hello, python!
   

3. strip(): Remove leading and trailing whitespace.

    # Remove whitespace
    message = "   hello, python!   "
    print(message.strip())
   
    # Output:
    hello, python!
   

4. replace(): Replace a substring with another substring.

    # Replace substring
    message = "hello, python!"
    print(message.replace("python", "world"))
   
    # Output:
    hello, world!
   

5. split(): Split a string into a list of substrings.

    # Split string into list
    message = "hello, python!"
    words = message.split()
    print(words)
   
    # Output:
    ['hello,', 'python!']
   

6. join(): Join elements of a list into a string.

    # Join list into string
    words = ["hello", "python"]
    message = " ".join(words)
    print(message)
   
    # Output:
    hello python
   

7. find(): Find the first occurrence of a substring.

    # Find substring
    message = "hello, python!"
    position = message.find("python")
    print("Position of 'python':", position)
   
    # Output:
    Position of 'python': 7
   

8. count(): Count the number of occurrences of a substring.

    # Count occurrences of substring
    message = "hello, python! python is fun!"
    count = message.count("python")
    print("Occurrences of 'python':", count)
   
    # Output:
    Occurrences of 'python': 2
   

9. capitalize(): Capitalize the first character of the string.

    # Capitalize first character
    message = "hello, python!"
    print(message.capitalize())
   
    # Output:
    Hello, python!
   

10. title(): Capitalize the first character of each word.

    # Capitalize first character of each word
    message = "hello, python!"
    print(message.title())
    
    # Output:
    Hello, Python!
    

11. swapcase(): Swap the case of all characters in the string.

    # Swap case of characters
    message = "Hello, Python!"
    print(message.swapcase())
    

Escaping Characters

Escaping characters in Python allows you to include special characters in strings that would otherwise be difficult or impossible to include directly. This is done using a backslash (\) followed by the character you want to escape.

# 1. Using a newline character(\n)
print("Hello\nSuraj")
#Output:
Hello
Suraj

# 2. Tab Character(\t)
print("Hell0\tSuraj")
# Output:
Hello    Wolrd

# 3. Backslash Character(\\)
print("This is a backslash: \\")
# Output:
This is a backslash: \ # This is the output because, The \\ sequence inserts a single backslash.

# 4. Single Qoute(\') and Double Qoute(\")
print('I said, \'This is single qoute!\'')
print("I said, \"This is double qoute!\"")
# Output:
He said, 'Hello, World!'
He said, "Hello, World!"

# 5. Carriage Return(\r)
print("Hello\rBro")
# Output:
Bro # The \r character moves the cursor to the beginning of the line, so "Bro" overwrites "Hello.

# 6. Bell/Alert(\a)
print("Hello\aBro")
# Output: You might hear a beep sound, and the text "HelloBro" will be printed. BecauseThe \a character triggers the system bell (alert).

String Characters & Slicing

A string is a sequence of characters. Each character in a string can be accessed using its index. Python strings are zero-indexed, meaning the first character has an index 0, the second character has an index 1, and so on.

text = "Python"
print(text[0])  # Output: P
print(text[1])  # Output: y
print(text[-1]) # Output: n (negative indexing starts from the end)

String Slicing

String slicing allows you to extract a portion of a string by specifying a start and end index. The syntax for slicing is: string[start:stop:step]

Examples of Slicing

# Basic Slicing
text = "Python"
print(text[1:4])  # Output: yth

# Omitting Start or Stop
text = "Python"
print(text[:4])  # Output: Pyth (Omits the start index, starts from the beginning)
print(text[2:])  # Output: thon (Omits the end index, goes till the end)

# Using Step
text = "Python"
print(text[::2])  # Output: Pto (Takes every 2nd character)

# Negative Indices and Slicing
text = "Python"
print(text[-5:-1])  # Output: ytho (From index -5 to -2, -1 is not included)
print(text[::-1])   # Output: nohtyP (Reverses the string)

# Reversing a String
text = "Python"
reversed_text = text[::-1]
print(reversed_text)  # Output: nohtyP

Booleans

# Simple program with booleans
is_active = True
print("Is active:", is_active)

# Output:

Build-In Functions

# Simple program using built-in functions
message = "Hello, Python!"
print("Length of message:", len(message))

# Output:

Enums

from enum import Enum

# Define an enum for days of the week
class Day(Enum):
    MONDAY = 1
    TUESDAY = 2
    WEDNESDAY = 3
    THURSDAY = 4
    FRIDAY = 5
    SATURDAY = 6
    SUNDAY = 7

# Accessing enum members
print(Day.MONDAY)            # Output: Day.MONDAY
print(Day.MONDAY.name)       # Output: MONDAY
print(Day.MONDAY.value)      # Output: 1

# Iterating over enum members
for day in Day:
    print(day.name, day.value)

# this is the Output:
"""
MONDAY 1
TUESDAY 2
WEDNESDAY 3
THURSDAY 4
FRIDAY 5
SATURDAY 6
SUNDAY 7
"""

User input

# Simple program with user input
name = input("Enter your name: ")
print(f"Hello, {name}")

# Output:
# Enter your name: Suraj # In console
# Hello, Suraj # actual output after entring name

Logic and Bit operators

Operators

# Simple program with operators
result = 10 + 5
print("Result of addition:", result)Logic and Bit operators

Operators

# Simple program with operators
result = 10 + 5
print("Result of addition:", result)

# Output:

Arithmetic Operators

# Simple program with arithmetic operators
quotient = 10 / 2
print("Quotient:", quotient)

# Output:

Comparison Operators

# Simple program with comparison operators
is_equal = (10 == 10)
print("10 is equal to 10:", is_equal)

# Output:

Boolean Operators

# Simple program with boolean operators
result = True and False
print("True and False:", result)

# Output:

Bitwise Operators

# Simple program with bitwise operators
bitwise_result = 5 & 3
print("5 AND 3:", bitwise_result)

# Output:

is & in Operators

# Simple program with 'is' and 'in' operators
x = 5
y = 5
is_same = (x is y)
contains = 'a' in 'apple'
print("x is y:", is_same)
print("'a' in 'apple':", contains)

# Output:

Ternary Operator

# Simple program with ternary operator
x = 10
result = "Even" if x % 2 == 0 else "Odd"
print("x is:", result)

# Output

Control Flow - Conditional Blocks & Loops:

Conditional Statements:

if Statement: Executes a block of code if a specified condition is true.

# Simple if statement
x = 10
if x > 0:
    print("Positive")

# Output:
Positive

if-else Statement: Executes one block of code if a condition is true, and another block of code if the condition is false.

# Simple if-else statement
x = -5
if x > 0:
    print("Positive")
else:
    print("Non-positive")

# Output
Non-positive

if-elif-else Statement: Executes different blocks of code based on multiple conditions.

# Simple if-elif-else statement
x = 0
if x > 0:
    print("Positive")
elif x < 0:
    print("Negative")
else:
    print("Zero")

# Output:
Zero

Loops:

while Loop: Repeats a block of code as long as a specified condition is true.

# Simple while loop
count = 1
while count <= 5:
    print("Count:", count)
    count += 1

# Output:
Count: 1 
Count: 2 
Count: 3 
Count: 4 
Count: 5

for Loop: Iterates over a sequence (such as a list, tuple, or string).

# Simple for loop
for i in range(5):
    print("Iteration:", i)

# Output:
Iteration: 0
Iteration: 1
Iteration: 2
Iteration: 3
Iteration: 4

Arithmetic Operators

# Simple program with arithmetic operators
quotient = 10 / 2
print("Quotient:", quotient)

Comparison Operators

# Simple program with comparison operators
is_equal = (10 == 10)
print("10 is equal to 10:", is_equal)

Boolean Operators

# Simple program with boolean operators
result = True and False
print("True and False:", result)

Bitwise Operators

# Simple program with bitwise operators
bitwise_result = 5 & 3
print("5 AND 3:", bitwise_result)

# Output:

is & in Operators

# Simple program with 'is' and 'in' operators
x = 5
y = 5
is_same = (x is y)
contains = 'a' in 'apple'
print("x is y:", is_same)
print("'a' in 'apple':", contains)

# Output:

Ternary Operator

# Simple program with ternary operator
x = 10
result = "Even" if x % 2 == 0 else "Odd"
print("x is:", result)

# Output

Control Flow - Conditional Blocks & Loops:

Conditional Statements:

1. if Statement: Executes a block of code if a specified condition is true.

    # Simple if statement
    x = 10
    if x > 0:
        print("Positive")
   
    # Output:
    Positive
   

2. if-else Statement: Executes one block of code if a condition is true, and another block of code if the condition is false.

    # Simple if-else statement
    x = -5
    if x > 0:
        print("Positive")
    else:
        print("Non-positive")
   
    # Output
    Non-positive
   

3. if-elif-else Statement: Executes different blocks of code based on multiple conditions.

    # Simple if-elif-else statement
    x = 0
    if x > 0:
        print("Positive")
    elif x < 0:
        print("Negative")
    else:
        print("Zero")
   
    # Output:
    Zero
   

Loops:

1. while Loop: Repeats a block of code as long as a specified condition is true.

    # Simple while loop
    count = 1
    while count <= 5:
        print("Count:", count)
        count += 1
   
    # Output:
    Count: 1 
    Count: 2 
    Count: 3 
    Count: 4 
    Count: 5
   

for Loop: Iterates over a sequence (such as a list, tuple, or string).

# Simple for loop
for i in range(5):
    print("Iteration:", i)

# Output:
Iteration: 0
Iteration: 1
Iteration: 2
Iteration: 3
Iteration: 4

Thanks for reading...

What are built and package manager tools?

When developers create an application, it needs to be available to end users. This involves deploying the application on a production server, which requires moving the application code and its dependencies to the server. This is where build and package manager tools come into play.

• Package application into a single movable file

• And this single movable file is called an artifact.

• Packaging = “building the code”

• Building the code involves:

  • Compiling (Hundreds of files into 1 single file)

  • Compress (Hundreds of files into 1 single file)

  • After the artifact was built we kept artifacts in storage also

  • In case we need to deploy multiple times, have a backup, etc.

    E.g.: If we deploy the artifact on the dev server, we deploy the test environment, or later on the production environment.

Artifact repository:

The storage location where we keep the artifact once we build it is called the artifact repository, an example is Nexus, JFrogArtifactory.

What kind of file is in the artifact?

• The artifact file looks different for each programming language

• E.g., if you have a Java Application the artifact file will be a JAR or WAR file

• JAR = Java Archive

• These include whole code and dependencies like:

  • Spring Framework

  • datetime libraries

  • Pdf processing libraries

Install Java and Build tools

• Install java

• Install Maven

• install Node + npm

• Download IntelliJ or VSCode

How to build the artifact with dependencies?

• Using a Build Tool, so there are special tools to build the artifact

• These tools are specific to the programming language

• For Java:

  • There is Maven and Gradle

• For NodeJs Or JavaScritp

  • There are npm and yarn

• What These tools do:

  • Install dependencies

  • Compile and compress your code

  • And can do some other different tasks

Build tools in Java

• JAR or WAR file

• Maven: Uses XML for configuration and is widely used for managing dependencies and building projects.

• Gradle: Uses Groovy for configuration and is known for its flexibility and performance.

• Which is more convenient for scripting and writing all those tasks for building the code or compiling etc.

• Both have command line tools and commands to execute the tasks

Gradle commands to execute the Java file (Doing Hands-On)

This command will compile the code, resolve dependencies, and package the application into an artifact using Gradle.

./gradlew build

In the Gradle, we don’t need to configure how the JAR file is built.

Maven command to execute the Java file

Add this plugin to your pom.xml file:

<build>
  <plugins>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
      <version><!-- Specify version here --></version>
      <executions>
        <execution>
            <goals>
               <goal>repackage</goal>
            </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

The command will compile the code, run tests, package the application into an artifact, and install it into the local Maven repository.

mvn install

Build Tools for Development (Managing Dependencies)

• It would help if you had the build tools also locally when developing the App

• Because you need to run the app locally

• run tests

Maven and Gradle have their own dependency files

• Dependencies file = Managing the dependencies for the project

Where these Dependencies files lies:

• For the maven its - pom.xml

• For the Gradle it's - build.gradle

You need a library to connect your Java application to ElasticSearch

• Find a dependency with name and version

• You add it to the dependencies file (e.g. pom.xml)

• Dependency gets downloaded locally (e.g local maven repo)

Run the application

Now let’s say we’ve built an artifact, stored it in the artifact repository, and copied it also on the server.

Now the question is how do you run the application with the artifact

This is the command to execute the .jar file for example:

// This example is for gradle
java -jar <name of jar file>

//This is an example of Gradle
java -jar build/libs/java-app-1.0-SNAPSHOT.jar

// This example is for Maven
java -jar <name of jar file>

//This is an example of Maven
java -jar target/java-maven-app-1.0-SNAPSHOT.jar

So if you are on a fresh server where and you have Java installed you can download the artifact from the repository and start the application using above commands.

Build JavaScript application

What about the JavaScrip Application

• JS doesn’t have a special artifact type

• So it can’t be built in ZIP or TAR file

So how do we build a js artifact file as a zip or tar file

These are the alternatives over Gradle and Maven in JavaScript

• npm - much more widely used

• yarn

These both contain package.json file for dependencies

npm and yarn are package managers and NOT build tools

These package managers install dependencies, but not used for transpiling JavaScript code or JavaScript artifact.

npm repository for dependencies : It refers to the npm registry, which is a public repository of JavaScript packages. It is used to find and install dependencies (libraries or modules) required for JavaScript projects.

Command Line Tools - npm

• npm start - start the application

• npm stop - stop the application

• npm test - run the test

• npm publish - publish the artifact

What does the zip/tar file include?

• application code, but not the dependencies

When you Run the app on the server

• You must install the dependencies first

• Unpack zip/tar

• Run the app

You need to copy the artifact & package.json file to the server to run the application

To create an artifact file from our node js application

npm pack

• JavaScript world is much more flexible than the Java world

• But not as structured and standardized

So the above is about nodejs a backed application but what about frontend application when it comes to package.

Package Frontend Code

Different ways to package this

• Package fronted and backend code separately

• Common artifact file

For example, if we have reactjs and nodejs application both in Javascript than we can have:

• Separate pakcage.json file for frontend and backend

• Common package.json file

• Frontend/react code needs to be transpiled!

• Browser don’t support latest JS versions or other fancy code decorations, like JSX

• The code needs to be compressed/minified!

• Separate tools for that - Build Tools/Bundler!

  e.g.: webpack

Build Tools - Webpack

Webpack is a build tool that bundles, transpiles, and minifies JavaScript and other assets for frontend applications.

It will transpiles, minified, bundles, compresses the code.

// install the webpack
npm install

// It is used to bundle or compress the javascript code 
npm run build

Dependencies for Frontend Code

• npm and yarn for dependencies

• Separate package.json vs. common package.json with backend code

• Can be an advantage to have the same build and package management tools

In case react has Java as a backend:

• Bundle-fronted app with webpack

• manage dependencies with npm or yarn

• package everything into a WAR file

What are Build tools for other programming languages

• These concepts are similar to other languages

• pip package manager for python

Publish an artifact into the repository

You build the artifact you need to push it to the artifact repository

• Build tools have commands for that

• Then you can download (curl, get) it anywhere

NOTE: we don’t fetch or push the artifact like that because of Docker.

The two important things to understand related to these package managers and build tools that we use for different programming languages Docker and another CI/CS pipeline(Jenkins)

Build tools & Docker

With Docker, the need to create zip or tar files is reduced. Docker images can directly contain the application code, simplifying the deployment process.

• With docker, we don't need to build and move different artifact types.

• There is just one artifact type - Docker images

• Now, we build Docker images from the application and we don't need a repository for each file type, no need to move multiple files to the server like package.json, no need to zip, just copy everything into the docker filesystem and run it from docker image.

• Docker image is also an artifact

• In order to start the application you don't need npm or java on the server, execute everything ( command to run the JAR or node application) inside the docker image.

NOTE: For the different Java and javascript applications - we don’t need to create zip or tar files anymore because of docker images because we can directly copy and paste the JS code files into docker image but we have to build the application

Docker file for Javascript

Docker file for java

Build Tools for DevOps Engineers

Why should you know these Build Tools?

DevOps engineers need to understand build tools to assist developers, automate builds, and manage CI/CD pipelines. This includes:

• Developers install dependencies locally and run the application, but don’t build it locally

• You need to execute tests on the build servers

  npm/yarn test

  gradle/mvn test

• Help Developers for building the application

• Because you know where and how it will run

• Build Docker image ⇒ Push to Repo ⇒ Run on Server

• You need to configure the build automation tool CI/Cd pipeline

• Install dependencies ⇒ run tests ⇒ build/bundle app ⇒ push to repo

NOTE: You don’t run the app locally

• Linux basics

• Command line basics

• No programming knowledge required

What is a Kernel?

Interface between hardware and software

What is a Bash/Shell?

• Bash: A command language interpreter used for interacting with a computer from the command line.

• It is like a container

• The interface between users and Kernel

• Also Known As Commonly referred to as "the shell."

• CLI is a shell

• Purpose:

  • Provides a user-friendly interface to interact with the operating system.

  • Hides the complex details of the operating system kernel.

  • Enables tasks such as accessing data and writing files through simple commands.

• Historical Significance:

  • Introduced in the early 1970s.

  • Revolutionized how users interact with computers.

• When you open any operating system like macOS or Linux's any distro like Ubuntu the default Shell is usually Bash.

• Bash is also a programming language that allows us to write scripts which means anything we manually type into the command line can be automated with code.

Find your Shell

•       echo $0
  

• Available Shells “cat /etc/shells”

    cat /etc/shells
  

  • Your Shell? /etc/passwd

    cat /etc/passwd

• Windows GUI is a shell

• Linux KDE(K Desktop Environment.) GUI is a shell

• Linus sh, bash, etc. is a shell

What is Bash/Shell Scripting?

A bash script is a file that contains one or more commands that are meant to be executed within your bash shell.

Why Shell Scripting?

Shell scripts in Linux are an easy-to-use time-saving solution for automating repetitive or complex sets of tasks in a system administrator’s daily life. It can help save time and increase productivity, so it’s a must-have skill for anyone working on computers.

• Automate daily backups

• Automate installation and patching of software on multiple servers

• Monitor system periodically

• Raise alarms and send notification

• Troubleshooting and Audits

• Many more

Types of Linux Shell

Below both of these are the desktop GUI environment

• Gnome

• KDE (K Desktop Environment)

CLI environment:

• sh - So, this just sh is a Bourne shell called sh, one of the original Linux shells. It is developed by Unix computers by Stephon Bourne. He worked for AT&T Labs in 1977. It offers features such as:

  • input and output redirection

  • shell scripting with string

  • integer variable and condition testing and looping etc…

• bash - It means Bourne again shell so it

• csh and tcsh - used specifically in cases if you are proficient with c/c++ languages. tcsh doesn’t run bash.

• ksh - KornShell( developed by David Korn). Korn shell is compatible with shell and bash shell as well. Korn Shell improves on Bourne Shell by adding floating point automatic job control.

Best practice to follow:

• Give your script a name that makes sense.

• Design your script to be reusable

• The script should not require to be edited before running

• Use command line arguments to pass inputs.

• For example(good naming) create-and-launch-rocket

• Bad Naming: script1.sh, myscript2.sh, test.sh, etc…

• Leave out .sh extension for executables:

• good: create-launch-rocket

• bad: create-launch-rocket.sh

Writing our first bash script:

#!/bin/bash

# A script that will print out a message

echo 'Hello Bash bro'

• #!/bin/bash: this line is called a ‘shebang’. This is the line from where the script starts to execute.

• (hashtag or pound or number sign) !(exclamation mark) These together mean that hey, this is a shebang.

• Shebang is used to tell the environment we’re in, and which interpreter we will be using.

• In this case, since the shebang is /bin/bash, we’re telling it, hey, we’re going to use bash which makes sense because this is a bash shell.

• #message: with # (comment) is used to document your script, and provide clarity in context to what you’re creating here/what our program will do.

• echo ‘Hello Bash bro’: to print the output.

How to execute a script

• First, make a file with the name you like. e.g.

    vi myscript
    OR
    vim myscript
  

• Give you file permission to execute:

    chmod a+x myscript
    # here a means to all users and x means execute
  

• Execute Script Directly:

  Run ./myscript to execute the script.

What is a variable and how do we use them

A variable is a way to store data and use it within our bash script.

Why would you want to do this?

• Well, it makes us more efficient because we’re repeating less code.

Variable - whose values can change

Constants - whose value can not change

• Variable only contain Alphanumeric OR Underscores

• And it’s Case Sensitive as well

Best practices:

• good one: mission_name

• bad one: Mission_Name, Mission Name, Mission-name

#!/bin/bash

# This is when we assign value to a variable
hello_message='Hello, world!'

# If we want to print the current directory along with the message 
current_dir=$(pwd)

#  And this is when we are using variable
echo "$hello_message from $curret_dir"

What are the positional arguments/parameters in shell scripting?

The positional parameters hold the arguments passed to a script or function. They are referenced using $1, $2, $3, etc., where $1 is the first argument, $2 is the second, and so on. $0 refers to the script's name.

Example script:

#!/bin/bash 

echo "First argument: $1" 
echo "Second argument: $2

If the script is executed with ./myscript arg1 arg2,
it will output: First argument: arg1 Second argument: arg2

How to accept user input

#!/bin/bash

echo Hello, My name is Suraj.
echo
echo What is your name?
echo 
read broname
echo
echo Hello $broname

read -p "Enter mission name:" variable_name
echo $variable_name

# read take the input 
# -p display the text

What are the conditional statements in bash?

Conditionals are just if else statements we can understand by the below example:

A simple if-then code in bash

#!/bin/bash

number=25

if [ $((number % 2)) -eq 0 ];
then
   echo "The number $number is even!";
else 
   echo "The number $number is odd!"
fi

#!/bin/bash

clear

if [ -e /home/devopsbro/basicbashscript/bro.txt ]

                then
                echo "File exists"
                else
                echo "File does not exist"
fi

What are Case Statements in Bash?

Most of the installation programs are written in case statements where the scripts wait for user input to select from choices.

#!/bin/bash

# A script that will ask for a number and print out a message depending on the value. 

read -p "Enter a value: " n
case $n in
    ???) 
        echo "One";;
    2) 
        echo "Two";;
    aa) 
        echo "Three";;
    *.txt) 
        echo "Four";;
    *) 
        echo "Other";;
esac

# read: it is the command to ask the user to provide a number

# -p: allows us to provide a prompt which in this case is enter a value

# n: name of the variable

What are loops in Bash?

• Program to automate the repeating task

• Here we are learning to do while done keywords

While loop

#!/bin/bash

# A script to display a series of numbers using a while loop.

counter=1
# here ‘ -le ‘ is less then
while [[ "$counter" -le 10 ]]; do
    echo "The counter is at: $counter"
    counter=$((counter + 1))
done
echo "The count has finished."

Until loop

#!/bin/bash

# A script to display a series of numbers using while.

counter=1
until [[ "$counter" -gt 10 ]]; do
    echo "The counter is at: $counter"
    counter=$((counter + 1))
done
echo "The count has finished."

for loop traditional

#!/bin/bash

# Print values in an array using for loops

services=("loadbalancer" "virtualmachine" "storage")

# here @ mean all the elements of the array services
for i in "${services[@]}"
do
   echo $i
done

for loop new

#!/bin/bash

# A script to display a series of numbers using a for loop.

for (( i=0; i<5; i=i+1 )); do
    echo "The counter is at: $i"
done

loop and break

#/bin/bash

# A script that will recieve input and break depending on condition. 

while true; do
# here n is variable that store the user input
  read -p "Enter a number between 1 and 25: " n
  if [[ $n -ge 1 && $n -le 25 ]]; then
    echo "You entered $n"
  else 
    echo "You didn't enter a number in range, goodbye."
    break
  fi
done
echo "Break happened"
# the ‘fi’ here is the opposite of ‘if’ to stop if statement

What are the functions in bash?

A function in Bash is a reusable block of code that can be called with a name to perform a specific task, defined using the syntax function_name() { commands; }.

#!/bin/bash

# function name and definition
check_even () {
    local mod=2
    echo "The value of mod is $mod"
    if [ $(("$1" % $mod)) -eq 0 ]
    then 
       echo "The number $1 is even!";
    else 
       echo "The number $1 is odd!"
    fi
}
# Variable Declaration
number=2344

# Function Call
check_even $number
echo $mod

local: It is a scope here which means whatever is defined inside the function which {} can’t access outside but only inside.

Check Remote Servers Connectivity

A script to check the status of remote hosts

#!/bin/bash

# this is a simple script
ping -c1 192.168.1.1
        if [ $? -eq 0 ]
        then
        echo OK
        else
        echo NOT OK
        fi

# Change the IP to 192.168.1.235

#!/bin/bash

# This is a good one
# Don't show the output
ping -c1 192.168.1.1 &> /dev/null
        if [ $? -eq 0 ]
        then
        echo OK
        else
        echo NOT OK
        fi

#!/bin/bash

# This is the better one
# Define variable

hosts="192.168.1.1"
ping -c1 $hosts &> /dev/null
        if [ $? -eq 0 ]
        then
        echo $hosts OK
        else
        echo $hosts NOT OK
        fi

# Change the IP to 192.168.1.235

#!/bin/bash

#Multiple IPs
IPLIST="path_to_the_Ip_list_file"


for ip in $(cat $IPLIST)

do
   ping -c1 $ip &> /dev/null
   if [ $? -eq 0 ]
   then
   echo $ip ping passed
   else
   echo $ip ping failed
   fi
done

Aliases(alias)

In real life, aliases are short names of long ones to remember.

Aliases is a very popular command that is used to cut down on lengthy and repetitive commands

# Here are the some example of how you can make an alias of any command

alias ls="ls -al"
alias pl="pws; ls"
alias tell="whoami; hostname; pwd"
alias dir="ls -l | grep ^d"
alias lmar="ls -l | grep Mar"
alias wpa="chmod a+w"
alias d="df -h |awk '{print \$6}' | cut -cl-4"

Note: {print \$6} here before $6, we put backslash “\” because we don't want Linux to not think of $6 as a variable.

• To know what and how many aliases you have made use the command alias to know.

• To delete an alias

    unalias nameOfAlias
  

• To delete all the aliases at once

    unalias -a
  

User and Global Aliases

Creating User or Global Aliases.

• User = Applies only to a specific user profile

    User = /home/user/.bashrc
  

• Global = Applies to everyone who has an account on the system

    Global = /etc/bashrc
  

Note: So basically if we are creating an alias in the terminal without user-specific then it’ll stay only for that session and when you close and open the terminal you’ll not find the alias again. So to solve this we’ll be creating a user alias:

Now let's make the alias Global:

for me, this command works to add a global alias

sudo nano /etc/bash.bashrc

Shell History

You can know all the commands that you have run in your terminal by just running the command history .

history

If you want to scroll more command of history just run this command:

history | more

If you want to run any command from you history just remember the number of that command left side and run it this way:

!8

Note: The file where the history of your shell commands is saved and to know that run the below command:

/home/username/.bash_history

The Linux commands you'll use while working with Bash

1. ls --help : To know more about ls command.

2. echo --help : To know more about echo command.

3. man : An interface to the system reference manuals.

4. pushd : Adds a directory to the directory stack.

5. popd : Removes a directory from the stack and changes to it.

6. mkdir foldername : Create the folder with the name specified.

7. mkdir -p suraj1/suraj2/suraj3 : For creating parent directories.

8. touch filename : To create a file.

9. mv source_name destination_name : Used to move or rename files and directories from source_name to destination_name .

10. cp source_name destination_name : Copies source_name to destination_name, creating a duplicate file or directory.

11. rm file_name_you_want_to_delete : Used to remove files or directories.

12. rm -r file_name_you_want_to_delete : For recursive deletion

13. rmdir directory_name : Used to delete

14. cat file_name : Used to view to content of the file.

15. head file_name : This will display by default the first 10 lines of your content.

16. head -n 5 file_name : Now this will display the first 5 number of lines of your content because we have specified it using the ‘-n’ flag.

17. tail file_name : This will display by default the last 10 lines of your content.

18. tail -n 5 file_name : This will display the last 5 number of lines of your content.

19. more file_name : It will give the information page by page. and hit the space bar to go page by page.

20. less file_name : ‘less’ and ‘more’ are very similar just the less has a lot more options in its navigation.

21. grep --help : To know more about grep

Use of Wildcards:

• To find any files with an asterisk(*) and with ??:

• To print all the files with the uppercase and lowercase as well:

How to use ‘whereis’ command:

whereis file_name

whereis -b file_name

which file_name

How to use the find command:

find . -name "*.md"

The above command says to find in the current directory(.) all the name which has the .md extension

find /home -name file.txt

find /home -iname file.text

So here I mean it’ll remove the case sensitivity and show all the files whose name is file.txt regardless if it is in lowercase or uppercase

find . -type d

How to use the environment variables command:

As the name suggests the environment variables set several variables for your shell environment. Environment variables are important because applications, command line tools, and shell scripts use them to determine their configuration.

• Knowing how to view, create, and modify environment variables is a must if you were to learn the ways of the terminal.

The ‘env’ command:

this command will show you all the environment variables that are available currently.

env

• If we want to see just a single environment variable, we can use the ‘echo’ command:

These are the environment variables:

echo $SHELL
# OUTPUT: /bin/bash

echo $HOME
# /home/user

echo $USER
# user

echo $PATH
# (example): /home/devopsbro/.nvm/versions/node/v20.15.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

The command to create an environment variable is ‘export’:

export --help : To know more about export command.

export newVar=value
echo $newVar

# output: value

Important: The environment variable that we just created is only for the current session. And the question comes in :

“How do you persist environment variables across sessions?”

and “the answer to this question is what bash calls startup files.”

• So there are several locations for the startup files but two worth mentioning are the ‘global configuration’ which is located under the ‘etc’ profile, and if you were to make changes to this particular file, it would affect every user logged in to the machine or their bash session for every user logged in the machine.

ls /etc/profile

• Another one is the ‘.’ profile which exists underneath the user’s home directory

ls -a ~

# it will output: .bashrc or .profile kind of profiles

How to use Redirection:

Everything that you see on the terminal either comes from an output stream or an input stream. In bash, you can redirect or chain together that output using the redirection operator and the pipeline.

• One of the streams that command right to is the standard output:

    ls - l
  

• Using redirection operator now:

    ls -l > file_name.txt
  

• If you want to put the new content in a new like you want to append:

    touch file_name.txt
    ls -l >> file_name_in_which_you_want_to_apppent.txt
  

Standard error:

ls -l ./dir

# This will give an error

Here 2 is the file descriptor that indicates the standard error, and so it’s saying redirect standard error to this particular location.

ls -l ./dir 2> error.txt

Now, this is without a file descriptor:

ls -l ./dir > error.txt

# It just redirects to the terminal because I'm not redirecting standard error, I am trying to redirect the standard output

So here 2>&1 in this we’re saying stream one and stream two redirects to these files.

ls -l ./dir > error.txt 2>&1
or 
ls -l &> error.txt 

# The above two commands are the same as long as we're doing this is in the same directory but not append.

If we want to append we can do this:

ls -l >> error.txt 2>&1

How to use pipelines ( | ):

This is without the pipe symbol

ls -l /usr/bin

as we know what less command do this will show the content less and we can scroll using the spacebar

ls -l /usr/bin | less

This will find the echo in this directory

ls -l /usr/bin | grep echo

This will find and sort the content

ls -l /usr/bin | grep echo | sort

This will show the first content of this file.

ls -l file_name.txt | head -n 1

This will show the last line of the file.

ls -l file_name.txt | tail -n 1

How to modify file permissions in Bash:

How to view permissions:

ls -ltr or
ls -l

Command to change the owner for Ubuntu users:

sudo chown root file_name.txt
# Here chown is the command and root is the new owner and the file name whose owner you want to change.

sudo chown :root file_name
# This command is used for changing the group owner

chgrp owner_name_you_want_change file_name
# This is the another command to change the group owner

How to use the chmod command:

This command will add the execute permission to the file for user, group, and other

chmod +x file_name

chmod 444 file_name
# The 444 means here in the user,group,and other it will give permission to execute the file.

Afterward, executing ls -l file_name might show something like:

-r--r--r-- 1 user group 1234 Jul 9 10:00 file_name

This table will help you to understand better:

Best Resources from where I learned

YouTube Video Resources:

1. Bash in 100 Seconds By Fireship

2. Shell Scripting Zero To Hero || Shell Scripting for Devops & Cloud || Shell scripting for Beginners

3. Welcome to Bash for Beginners [1 of 20] | Bash for Beginners

Thanks for reading guys

The Linux Foundation Mentorship Program, commonly known as LFX Mentorship, LFX mentorship is previously known as community bridge. It's a platform designed by the Linux foundation that accelerates the adoption, innovation, and sustainability of open-source software. Basically Linux Foundation mentorship program is designed for developers mainly first-time open-source contributors learn, experiment, and contribute effectively in the open-source communities.

What is open-source?
Open source is like joining a creative community fair where you and others build awesome things together. You contribute your ideas and skills, learn new things, and collaborate like friends working on a big project. It's a global effort, making a positive impact on people's lives, and it's a joyful journey of growth and achievement.

open-source is everywhere(means there are code and non-code contributions it's not just for developers keep that in mind!).

In general, it's free to use, access, and modify depending on the licenses. It's an inclusive community and you get to collaborate with people around the world.

How do I get to know about LFX Mentorship?

Actually I get know about LFX Mentorship program from the tech communites/People on platforms like Twitter, Discord etc...

OpenSource is life-changing for a lot of folks. Companies are hiring OpenSource Engineers. LFX Mentorship is for folks starting out in OpenSource and interested in exploring the modern tech niches companies are looking forward to.

Watch this video for more:

Varies for each term(as per the LFX Website):

• (Spring) Mentorships available on LFX Mentorship: mid-January

• (Summer) Mentorships available on LFX Mentorship: mid-April

• (Fall) Mentorships available on LFX Mentorship: mid-July

There are only two criteria to get into the LFX Mentorship Program:

1. You have to be 18 years old.

2. Not to be a prior participant in the LFX mentorship program

3. Other than it's open to all.

Apply Here:

Explore Mentorship Programs

What is CloudForet?

Cloudforet is an Open Source Platform for Enabling Integrated Management of Multi-Clouds.

Read more about Cloudforet

My Journey at Cloudforet as a LFX Mentee’23

It was a great experience to be an LFX Mentee at Cloudforet because it allowed me to learn more about cloud and DevOps concepts.

I got selected for the Cloudforet plugin development mentorship program:

After the selection of Cloudforet LFX mentorship, I have been assigned to 5 Tasks to do:

• Week 1. Orientation and Fundamentals

  • So in this, I've been introduced to the organization and the installation of the Cloudforet with minikube in my local system(I used Ubuntu OS for this).

• Step 2. Understanding of Cloud Computing & Kubernetes

  • Learned about the fundamentals of cloud computing and Kubernetes and CI/CD using github actions also.

• Step 3. Understanding of Protobuf & gRPC

  • To understand Protobuf & gRPC is the base framework of Cloudforet the LFX certification course was assigned, and I learned about it.

• Step 4. Plugin Development Basic

  • I've been assigned to first install Cloudforet, and then create a user for the service. and then create an API user and then create an API Key for CLI.
    After that to registered the local plugin marketplace for myself. Then I can enable my custom plugin.

• Step 5. Real-World Plugin Project

  • Code Analysis of plugin-http-file-cost-datasource

  • In this, I've been assigned to do:

    • Updating the DataSource at Step 4.

    • change base_url with your own CSV file (using Spacectl CLI)

    • Report for sync API call flow

    • based on interface, service, manager layer

    • How cost-analysis service and plugin-http-file-cost-datasource interact

I graduated successfully from Cloudforet:

Cloudforet 1:1 mentorship by Choonho Son(Mentor)

While doing this mentorship program I was stuck most of the time in understanding the tasks and completing them so my mentor Choonho sir always helped me to understand and complete those tasks and I learned a lot from him.

Conclusion

My experience was great at Cloudforet as a LFX mentorship. And you should apply in the open-source programs like as soon as possible because I can give the this surity that you'll gain somethign new and valuable skills from these programs and networking( you'll meet amazing people from around the world who are working in these kind of organizations.

Check my blog on open-source

• During the Cold War, the United States and the Soviet Union competed to be the best.

• In 1957, the Soviet Union surprised everyone by launching Sputnik 1, the first man-made satellite.

• To keep up, the U.S. created ARPA (Advanced Research Project Agency) to stay ahead in science.

• ARPA made ARPANET, the first big computer network using packet switching.

• ARPANET laid the foundation for the internet we use today, changing how we connect and work together.

The U.S. Advanced Research Projects Agency Network (ARPANET) was the first big computer network. It started in 1969 and stopped in 1989. ARPANET was mainly for academics and research.

WWW came into the picture:

We wanted to easily share research papers, but couldn't because there was no automatic way to do it.

Then, Mr. Tim Berners-Lee created the World Wide Web (www). It lets us store and access documents online. However, back then, we couldn't search for things on the website.

This is the world's first website

• The World Wide Web, also called the Web, is a place on the Internet where we find documents and other stuff. Each thing has its special address called a URL, like https://example.com/. These things can link to each other, and we can access them through the internet.

Who is responsible for managing all this stuff:

The Internet Society is responsible for all the rules and regulations that we have to send our data over the Internet.

Client-Server Architecture:

This is the typical client-server model:

• Your computer/laptop can work as a client and server.

• The client-server model is a way computers communicate over networks.

• In this model, one computer acts as a client, requesting services or resources.

• Another computer acts as a server, providing those services or resources.

• These computers can be anywhere in the world, connected by wires laid under oceans.

• Your computer or laptop can act as both a client, requesting information, and a server, providing information, depending on the situation.

Submarine cables map (Optical Fibre Cable)

”So the internet is not like in the clouds but it’s under the sea”

How is data Transferred?

Packets: Packets are small chunks of data that travel across networks like digital envelopes. They have destination addresses and come together to deliver messages or files.

This is how the data is transferred:

1. Connected to the Internet: Devices connect via Wi-Fi, Ethernet, or mobile data.

2. ISP (Internet Service Provider): Provides internet access.

3. Modem/Router: Provided by the ISP, assigns global IP addresses.

4. Device1,2,3: Receives local IP addresses via DHCP for communication within the network.

5. IP (Internet Protocol): Global IP identifies devices on the internet; local IP identifies devices within a network.

6. Data Transfer: Devices package data into packets with destination IP addresses. Modem/router routes packets based on global IP. Destination devices receive and reconstruct data using local IP.