Cloud Services and Internet of Things
1 Overview
Introduction and course organization
1.1 Objective
Connect Internet of Things (IoT) enabled devices using scalable cloud services in a project setup.
What you will build
Devices that communicate with cloud services
Devices
- Each device consists of a hardware and a software part
- Usually, real hardware executes software that is loaded onto the hardware
- To simplify development, you can use a script
Cloud Services
- We use real cloud services
- Currently, we have education partnerships with: Google Cloud Computing
- Unfortunately GCP discontinued their IoT Core service :(
1.2 Project Ideas
You should come up with your own project idea, but here is some inspiration:
- đĄī¸ Temperature/Humidity Monitoring System
- đ Air Pollution Monitoring System
- đ Noise Level Monitoring System
- đ ŋī¸ Park Spot Monitoring System
- đŠē Health Monitoring System
- đĻ Traffic Management System
- đ Railroad Control
- đĒŠ Festival Crowd Control
- ...
You don't have to solve complex problems and can also create something funny
1.3 Constraints
- Start from scratch, don't use presets or generators
- Keep project dependencies to a minimum
- Keep it small and simple
- Don't forget tooling and versioning
Consider: The idea of your project does not matter so much, we want to learn workflow and technology here
1.4 Prerequisites
- No formal prerequisites
- It's technical, you will have to code
- Proficiency in a language of your choice
You should have heard of
HTTP, TCP/IP, UDP, SSH, WebSockets, JSON, Git, Docker, API, Continuous Everything, Pull/Merge Request
1.5 Links
Description | Link |
---|---|
Gitlab Repositories Team Projects, Source Code, Issues, ... |
https://gitlab.mi.hdm-stuttgart.de/csiot/ss25 |
Supplementary Code Repositories Code examples, emulator, ... |
https://gitlab.mi.hdm-stuttgart.de/csiot/supplementary |
Hybrid?
The course takes place in presence.
If you want to participate remotely (e.g. due to covid infection, quarantine or other legitimate reasons), write us an email early enough. We'll ensure to bring the Meeting Owl and start the BBB-Stream. It's not a high-quality hybrid setup though, just sound and a shared screen.
1.6 Schedule
Date | Session (14:15 - 17:30) | Description |
---|---|---|
18.03.2025 | Kickoff | Course overview, questions and answers |
25.03.2025 | Lecture + Idea Pitch | Pitch your project idea (if you have any). Do you already have a team? |
01.04.2025 | Lecture + Team Setup | Maximum of 6 teams (4-6 members) and 30 students in total Higher semesters take precedence over lower semesters |
08.04.2025 | Lecture | |
15.04.2025 | Lecture + Team Meetings | |
22.04.2025 | Lecture / Working Session | |
29.04.2025 | Working Session + Q&A | Questions regarding presentations at the beginning of the session |
06.05.2025 | Midterm presentations | |
13.05.2025 | Working Session + Team Meetings | |
20.05.2025 | Working Session | |
27.05.2025 | Working Session + Team Meetings | |
03.06.2025 | Working Session | |
17.06.2025 | Working Session + Team Meetings | |
24.06.2025 | Working Session + Q&A | Questions regarding presentations at the beginning of the session |
01.07.2025 | Final presentations | |
06.07.2025 (Sunday) | Project submission | Commits after the submission date will not be taken into account Make sure to check the submission guidlines |
Lecture Sessions
- We'll talk through this slides in presentations
- Cloud Computing and Services
- Internet of Things
- Hands On / Assignments
- Call for teams and projects
Working Session
- Working sessions start with the team meetings
- Every team has a fixed 15 minutes slot
- Prepare the meeting (what have you done, what are you planning, specific questions to problems, ...)
- You can use the remaining lecture time to work on the applications
- The remaining time after the team meetings:
- We can discuss general questions together
- We can help with specific problems of your projects
Team Meetings
- Every team has a fixed 15 minutes slot
- Prepare the meeting (what have you done, what are you planning, specific questions to problems, ...)
- Attendance is required
Group | Time |
---|---|
Team A | 14:30 - 14:45 |
Team B | 14:45 - 15:00 |
Team C | 15:00 - 15:15 |
Team D | 15:15 - 15:30 |
Team E | 15:30 - 15:45 |
Presentation Sessions
- Each member has to present equally
- Unexcused absence will make you fail the course
- Duration: 12 min per team
- Midterm presentations
- Introduce yourself (who are you, what are you studying, what's your background)
- Each team presents its current state
- Check the grading slide for general presentation rules
- Content: project idea, technology-stack, (maybe schedule, architecture, docker & code)
- Final presentations
- Each team presents its final state
- Check the grading slide for important things to present
- Open and present live in browser
- Explain architecture, devices, data flow, lessons learned
1.7 Project Submission
Grading is based on the Gitlab repo in the csiot group https://gitlab.mi.hdm-stuttgart.de/groups/csiot
Add a README.md
in the repo that contains:
- project name
- members (full name, student short, matriculation number)
- project abstract
- technical documentation
1.8 Grading
The total of 50 Points is split into 4 categories.
Following general best practices is required for each category.
Although the grades are derived from the team, each individual gets a distinct grade that can differ from the other team members.
Code & Architecture (20 Points)
- Naming is consistent and fits best practices of language
- Code comments make sense and help to understand the flow
- File/Folder/Package structure is clean and makes sense
- Most work is done using cloud services, e.g. AWS IoT Events, DynamoDB, ...
- At least 2 sensors are used
- At least 2 actuators are used
- Application contains events that depend on data from multiple devices
- Application is horizontally scalable (minimum configuration to add or remove devices/things)
Tooling (15 Points)
- Deployed code, e.g. lambdas, is tested
- Infrastructure setup is automated, e.g. with terraform
- Infrastructure can easily be setup by lecturers
Presentation (10 Points)
- Each team member has equal presentation time and content
- Each team member knows about all areas of the project
- Group highlights lessons learned
- Presentation is finished in the alloted time
- Presentation is well prepared and works
Technical documentation (5 Points)
- Contains a very short project abstract
- Contains setup instructions
- Contains an architecture diagram
- Contains a data flow diagram
1.9 Ask for help!
In general:
The Center for Learning and Development, Central study guidance, VS aka student government support you:
- Exam nerves, fear of failure, financial problems, stress, depression, ...
- Bullying, racism, sexism, discrimination, ...
- Tipps and feedback regarding scientific writing (e.g. bachelor thesis)
- Career options after the bachelor
- Support for decision-making
Regarding this course:
- Don't be afraid to ask questions about your project (that won't affect your grading negatively)
- Talk to us early if there are any problems within the group (someone never shows up or does not support the group)
1.10 Questions
- Do you know what you will build?
- Do you know how it is graded?
- Do you know what presentation, lecture, working, Q&A sessions etc. are?
- Anything else?
2 Cloud Computing and Services
Introduction
2.1 What is Cloud?
What do you think?
2.2 Definition
Cloud computing is the on-demand availability of computer system resources, without direct active management by the user.
2.3 Timeline
- 1960s-90s: Initial concepts by Compaq, AT&T, IBM, DEC, ...
- 2002: Amazon creates Amazon Webservices (AWS) and the Elastic Compute Cloud (EC2 2006)
- 2008: Google creates App Engine
- 2008: NASA creates OpenNebula (EU funded)
- 2010: Microsoft creates Azure Cloud
- 2010: NASA and Rackspace create OpenStack based on OpenNebula
- 2012: Google creates Compute Engine
- 2015: Cloud Native Computing Foundation CNCF Landscape
2.4 Service Models
Service | Description | Examples |
---|---|---|
Infrastructure as a service (IaaS) | High-level API for physical computing resources | Virtual machines, block storage, objects storage, load balancers, networks, ... |
Platform as a service (PaaS) | High-level application hosting with configuration | Databases, web servers, execution runtimes, development tools, ... |
Software as a service (SaaS) | Hosted applications without configuration options | Email, file storage, games, project management, ... |
Function as a service (FaaS) | High-level function hosting and execution | Image resizing, event based programming, ... |
What is: Amazon Elastic Compute Cloud (EC2)? Adobe Creative Cloud? DynamoDB? OpenFaaS? Vercel Functions? Google Firebase? iCloud? GitHub? Vercel? Microsoft 365? Zoom? Azure Virtual Machines? Amazon S3? Gmail? Hosted Kubernetes? Knative? AWS Lambda? Shopify? Google Compute Engine (GCP)? Dropbox?
2.5 Cost estimation
- Cloud providers usually offer tools for cost estimation: https://calculator.aws
- What is expensive? What is cost effective?
- What are the expenses to run our OneMillionPixels application using AWS IoT Core (simplified)?
You have to know your application, requirements and use-cases!
3 Cloud Applications
Introduction
3.1 What is a Cloud Application?
What do you think?
3.2 Definition
A software with an architecture that leverages a multitude of cloud services.
3.3 VideoApp
An example app and web platform that allows friends from all over the world to collaboratively create a movie from their holidays
3.4 VideoApp Features
- Users can sign up to the platform using an eMail or a third party provider
- Users can create holiday groups and invite friends
- Friends can upload raw footage into holiday groups and tag it
- Friends can edit the footage into a movie using an online editor
3.5 VideoApp Requirements
Feature | Technical elements |
---|---|
Users can sign up to the platform using an eMail or a third party provider | Email, OAuth2 provider, relational data storage, ... |
User can create holiday groups and invite friends | Relational data storage, caching, notification, ... |
Friends can upload raw footage into holiday groups and tag it | Relational data storage, object storage, transcoding, queueing, search index, caching, notification, ... |
Friends can edit the footage into a movie using an online editor | Object storage, transcoding, queueing, caching, notification, ... |
For development and operations: | System monitoring and alerting, distributed logging, automated integration and deployment, global content distribution network, virtual network, system environments (development, staging, production, ...) |
3.6 VideoApp Architecture
Feature: Friends can upload raw footage into holiday groups and tag it
4 Cloud Infrastructure
Introduction
4.1 Technical View
Conventional Infrastructure | Cloud Infrastructure |
---|---|
(Bare-metal) Servers, Type 1/2 Hypervisors, Containers | Cloud Resources |
Long-living assets | Short resource life span |
Own data center, Colocation, Rented dedicated servers | No own hardware |
Direct physical access | No access on hardware |
4.2 Organizational View
4.3 Challenges
Short-living resources
Deployment, configuration, maintenance and teardown has to be automated
DevOps
Developers need to understand the runtime environment
Operators need to understand some application layers
New components in the application stack
Service discovery, service configuration, authentification/authorization and monitoring
4.4 Advantages
Continuous everything
Integration, deployment, delivery
High availability
Scalability, reliability, geo replication, disaster recovery
5 Cloud Tooling
Introduction
5.1 Infrastructure as Code
A coded representation for infrastructure allocation and configuration
Options for Infrastructure Configuration
Name | Is IaC | Automatisation | Declarative | Vendor Agnostic |
---|---|---|---|---|
Web Interface | â | â | â | â |
CLI | â | â | â | â |
SDK | â | â | â ī¸ | â |
IaC Tools | â | â | â | â |
Terraform Example
Example: Creating a S3 Bucket on AWS using Terraform
provider "aws" {
access_key = "xxx"
secret_key = "xxx"
region = "eu-central-1"
}
resource "aws_s3_bucket" "terraform-example" {
bucket = "aws-s3-terraform-example"
acl = "private"
}
Pulumi Example
Example: Creating a S3 Bucket on AWS using Pulumi
import pulumi
from pulumi_google_native.storage import v1 as storage
config = pulumi.Config()
project = config.require('project')
# Create a Google Cloud resource (Storage Bucket)
bucket_name = "pulumi-goog-native-bucket-py-01"
bucket = storage.Bucket('my-bucket', name=bucket_name, bucket=bucket_name, project=project)
# Export the bucket self-link
pulumi.export('bucket', bucket.self_link)
Other Tools
- Packer
- Docker
- Cloud Init
- Ansible
- Chef
- Puppet
5.2 Continuous Everything
Continuous | Requires | Offers | Implementation |
---|---|---|---|
Integration | Devs need correct mindset Established workflows |
Avoids divergence Ensures integrity/runability |
Shared codebase Integration testing |
Deployment | Automated deployment Access control / Permission management |
Ensures deployability | Infrastructure as Code Docker Swarm, Kubernetes, Nomad, ... |
Delivery | Deployability Approval from Marketing, Sales, Customer care |
Rapid feature release cycles Small to no difference between environments |
Same as for continuous Deployment Release/Feature management |
5.3 Monitoring and Alerting
Proper Monitoring/Alerting is essential when CD is applied
AWS CloudWatch
Service for time series data, logs and dashboards
Grafana
Prometheus: Time series database, metric exporters, Alertmanager
Grafana: (Real-time) Dashboards for monitoring data, Alerting Engine
5.4 Backup and Restore
Backup
- Automatic backup of stateful components
- Backup location preferably on external system (e.g. AWS S3)
Restore
- Restore process needs to be defined and tested
- Important for disaster recovery, useful for migration tasks
6 Internet of Things
Introduction
6.1 What is the Internet of Things?
What do you think?
6.2 Definition
A system of interrelated computing devices that can transfer data over a network without human interaction
6.3 Architecture
6.4 Hardware
Type | CPU (Max) | RAM | OS | TCP/IP | GPIO | ||
---|---|---|---|---|---|---|---|
DHT22 | ![]() |
Sensor | - | - | - | â | â |
Arduino (ATmega328P) | ![]() |
MCU | 20 MHz 8-bit RISC | 2 KiB SRAM | - | â | â |
ESP32 (Xtensa LX6) | ![]() |
SoC | 2 * 240 MHz 32-bit RISC | 520 KiB SRAM | e.g. FreeRTOS | â | â |
Raspberry Pi 4 (ARM Cortex-A72) | ![]() |
SoC | 4 * 1.5 GHz 64-bit ARM | 4 GiB DDR4 | GNU/Linux | â | â |
Random Gaming-PC | ![]() |
PC | 8 (HT) * 5.0 GHz 64-bit x86 | 32 GiB DDR4 | e.g. GNU/Linux | â | â |
6.5 Protocols
The following protocols are often used in an Internet of Things stack
Name | Network Layer | Description |
---|---|---|
LoRa(WAN) | Layer 1/2 | Low power, long range, uses license-free radio frequencies |
ZigBee | Layer 1/2 | Low power, 2.4 Ghz, 64 bit device identifier |
6LoWPAN | Layer 1/2 | Low power, 2.4 Ghz/ license-free radio frequencies, IPv6 addressing |
Ethernet | Layer 1/2 | Frame based protocol, also used for the normal internet |
802.11 Wi-Fi | Layer 1/2 | Wireless local area network protocol, also used for the normal internet |
IPv4 and IPv6 | Layer 3 | Packet based protocol, also used for the normal internet |
Bluetooth LE | Layer 3 | Low energy, wireless personal area network protocol, different from normal bluetooth |
MQTT | Layer 7 | Lightweight, Message Queuing Telemetry Transport protocol, publish-subscribe model |
6.6 MQTT
Lightweight, publish-subscribe network protocol that transports messages between devices
MQTT (Version 5) is a OASIS standard and ISO recommendation (ISO/IEC 20922)
- A client sends a message to a topic, e.g. /sensors/temperature/garage
- A client can subscribe to multiple topics, e.g. /sensors/temperature/+
Mosquitto is a popular lightweight server (broker)
7 Other topics
7.1 Mutual TLS authentication (mTLS)
What is...
- Asymmetric cryptography
- A certificate authority
- A certificate
X.509
openssl x509 -in cert.pem -noout -text
- Interesting fields
- Issuer
- Subject
- X509v3 Extended Key Usage
openssl s_client -connect emqx.services.mi.hdm-stuttgart.de:8883
Important: Server and client certificate do NOT have to be signed by the same CA!
7.2 AWS account setup
- Work as a team on one AWS root account
- Grant access to your teammates using one of the following methods:
- IAM user â preferred for this course
- AWS Organizations SSO user (IAM Identity Center) â recommended for professional usage
- Follow the principle of least privilege (start with the policies you currently need, you can add more later)
You don't have to use IaC to manage your account and users
Example policy for IAM users:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:GetAccountPasswordPolicy",
"iam:ListUsers",
"iam:GetAccountSummary"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:GetLoginProfile",
"iam:UpdateLoginProfile",
"iam:ChangePassword"
],
"Resource": "arn:aws:iam::*:user/${aws:username}"
},
{
"Effect": "Allow",
"Action": [
"iam:DeleteAccessKey",
"iam:GetAccessKeyLastUsed",
"iam:UpdateAccessKey",
"iam:CreateAccessKey",
"iam:ListAccessKeys"
],
"Resource": "arn:aws:iam::*:user/${aws:username}"
}
]
}
8 Hands On
Live demos and practical exercises to learn some new technologies
8.1 Assignment 0: GIT repository
Before you start with the assignments, create a personal project in GitLab and invite us. Push your state regularly during the assignments and follow best practices (e.g. *.pem in .gitignore)
You should do all assignments yourself, of course you can help each other.
8.2 Assignment 1: One Million Pixels (MQTT)
OneMillionPixels is a public canvas, where everyone can draw using MQTT messages.
https://pixels.services.mi.hdm-stuttgart.de
Tasks
- Choose a programming language of your choice and get familiar with an MQTT client library (e.g. paho-mqtt for Python)
- Connect to the MQTT broker with your personal TLS keypair.
- Create a client that publishes pixel data to the given topic to draw in real-time on the one million pixels page.
Connection
Broker
- Host: emqx.services.mi.hdm-stuttgart.de
- Port: 8883
- Topic: csiot/pixels
- TLS: mutual
MQTT message schema
{
"type": "object",
"properties": {
"x": {
"type": "integer"
},
"y": {
"type": "integer"
},
"r": {
"type": "integer"
},
"g": {
"type": "integer"
},
"b": {
"type": "integer"
}
}
}
8.3 Assignment 2: AWS IoT Core (Cloud Console)
Get familiar with the AWS IoT Core service
- Getting started https://docs.aws.amazon.com/iot/latest/developerguide/iot-quick-start.html
- Understand the different resource types: https://docs.aws.amazon.com/iot/latest/developerguide/iot-gs-first-thing.html
- Build your own solution:
- Connect to the broker without the AWS SDK
- Create a sender script that publishes random data to:
/csiot/${device_name}/temperature
- Create a receiver script that subscribes to:
/csiot/*/temperature
- Ensure that policies allow access
- You can publish further data e.g. humidity and choose a more generic subscription to read all values (checkout the mqtt subscription syntax)
8.4 Assignment 3: Terraform Basics
TLS based client-authentication with own CA
Build and maintain your own custom Certificate authority for client-authentication using Terraform.
Reguirements:
- Terraform: https://www.terraform.io/downloads
Tasks
- a) Discuss: What is the role of a CA? Why do we need it in the context of IoT device authentication?
- b) Generate/Apply CA/CSR's/Certificates using the provided solution (adjust for your needs): https://gitlab.mi.hdm-stuttgart.de/csiot/supplementary/handson/-/tree/master/solutions/04-terraform-tls-ca
- c) Use
openssl
to inspect the generated certificates, e.g.openssl x509 -in a-cert.pem -noout -text
- Who signed this certificate? For what can this certificate be used?
- d) Use GitLab to store Terraform's state to work as a team on the same CA.
Resources
8.5 Assignment 4: AWS IoT Core Devices with Terraform
8.6 Assignment 5: Pimoroni Enviro+ (Gateways software)
- Gateway Software for physical device