Course Information

Language: English
Type: 3V + 2Ü
Module: IN0006
ECTS Credits: 6
Prerequisites: Lab Course – Foundations of Programming (IN0002)
- Only students, who passed IN0002 or a comparable course, can participate in this course
- You must have experience with object oriented programming in Java
TUM Online: You must register for this course in TUM Online before the course starts
Contact:
- Post your questions to the corresponding channels on eist2023.slack.com .
- You can contact us at eist23.dse@xcit.tum.de. Do not contact instructors on their personal email addresses!
Time:
- Every Thursday: 08:15-11:00 (see the exact dates in the lecture schedule)

Important Information

You must have your own computer, and you are required to use it and during the tutor groups!
You must have experience with object-oriented programming in Java and with an IDE (e.g., Eclipse, IntelliJ)

Intended Learning Outcomes

After successful completion of this module, students are familiar with the basic concepts and methods of the different phases of a project, e.g. modelling the problem, reuse of classes and components, and delivery of the software. They have the ability to select and apply suitable concepts and methods for concrete problems.

The students know the most important software engineering terms and workflows and are able to analyze and evaluate given problems. In addition, students can solve concrete problems in software engineering, e.g. with the help of design patterns.

Content

Software engineering is the establishment and systematic use of engineering principles, methods, and tools for the division of work, the development, and application of extensive, complex software systems. It deals with the development and production of software, the organization and modeling of data structures and objects, and the operation of software systems.

Almost all modern software systems are designed for and deployed in the cloud. The new curriculum will focus on “software engineering for the cloud.” The lecture will cover ten major topics, among others:

L01 Introduction

Part I: Administrative
- Introduction to course staff and tutors
- Understand the organization of lectures, tutorials, exercises, and exams
- Introduction to course tools such as Artemis and Slack
Part II: Introduction to software engineering
- Explain the basic terminology: software engineering and activities
- Abstraction
- System modelling with UML
Part III: Course overview
- Course focus: Cloud software engineering
- An overview of software engineering activities in the cloud
Part IV: Software engineering models
- Defined (e.g., Waterfall model) and agile methods (e.g., Scrum)
- Apply agile methods in software engineering projects (Scrum)
- A note on tactical vs. strategic programming

L02 Cloud Systems Engineering

Part I: Introduction to cloud computing
- What is cloud computing?
- Why is it important for software engineering?
- Types of cloud services
  - IaaS, PaaS, and SaaS
- Types of cloud infrastructure
  - Public, private, and hybrid cloud
Part II: Cloud hardware architecture
- Data center and cloud infrastructure
- Virtualization:
  - Compute
  - Storage
  - Network
Part III: Cloud software systems
- Distributed systems
- The role of distributed systems in cloud computing
- Fallacies of distributed systems — challenges of distributed systems
- Remote procedure calls (RPC) via gRPC
- Serialization and deserialization of structured data using Protbuf
- Introduce a simple client-server application as an example distributed system
Part IV: Software deployment models in the Cloud
- Workflows, advantages, challenges, use-cases, examples, best practices
  - Baremetal
  - Virtual machines
  - Containers
  - Serverless
Part V: “Hello World in the Cloud” w/ Container
- Understanding the design, implementation, and deployment of a simple cloud application
- Introducing aspects of container-based (Docker) application development and deployment

L03 Cloud Design Requirements and Software Architectures

Part I: System design requirements
- Requirements engineering
- Stages in requirements engineering
- Functional vs non-functional requirements
- Non-functional requirements
  - Scalability
  - Reliability
  - Performance
  - Security
  - Maintainability
  - Deployability
Part II: Software architectures in the cloud
- Three-tier architectures
  - Definition of tiers
  - Communication layer
    - REST
- Monolithic architectures
  - Architecture overview
  - Deployment models
  - Advantages
  - Limitations
Part III: Microservices
- Microservice architectures
- Advantages/drawbacks of microservice architectures
- How to design and build a simple microservice application
- How to deploy a simple microservice application w/ Kubernetes
- Google’s Service Weaver: A Framework for Writing Distributed Applications

L04 System Design and Implementation I

Part I: System design challenges
- Software complexity and the quest for simplicity
- Design goals
- System design trade-offs
- Hints for computer system designs
Part II: Modularity
- System decomposition into sub-systems/components
- Subsystem decomposition: Modules
  - Create an initial subsystem decomposition
  - Differentiate between coupling and cohesion
- Pattern implementation:
  - Facade pattern
- Interface design
  - Shallow vs deep modules: The trade-offs between interface and functionality
  - General purpose modules are deeper
  - Information hiding (and leakage) principle
Part III: Software architecture: Layered architectures
- Subsystem decomposition: layered architectures
  - Ubiquitous adoption of layered architectures in systems
  - Open vs closed layered architectures
  - Different layers, different abstraction
  - Pulldown the complexity downward
Part IV: Data management
- Data management systems
- Distributed data management systems
  - A generic controller-worker architecture for distributed data management systems
- Key-value stores (KVS)
  - In-memory KVS
  - Persistent KVS
  - Distributed KVS w/ sharding
- Filesystems
  - Linux filesystems
  - Distributed filesystems
- Shared log
  - Shared log abstraction
  - Distributed shared log systems
- Databases:
  - Relational databases and SQL
  - Transactions
  - Distributed databases and distributed transactions
- Data management in the cloud:
  - Data warehousing with Data Lakes

L05 System Design and Implementation II

Part I: Security
- Security engineering
  - Security policies: Threat model and security (CIA) properties
  - Security design principles
    - Least privilege
    - Compartmentalization
    - Isolation via privilege mediation
  - A general recipe for secure system design
  - Access control
    - Access control lists (ACLs)
    - Capabilties
- Software security in the cloud
  - Security challenges in the cloud
  - Secure systems stack
    - Compute
    - Network
    - Storage
  - Authentication, key management, and attestation
    - Identity and access management (IAM)
    - Key management service (KMS)
    - Remote attestation
Part II: Reliability and availablity
- Terminology: System failures, fault types and sources, and properties and metrics
- Single-node fault-tolerant systems
  - Write-ahead logging for system reliability
  - Issues with a logging-only approach
- Replication as the general recipe
  - Replication for stateless services
  - Issues with replication for stateful services
- Replication for stateful services
  - Primary-backup replication
  - State machine replication
Part III: Performance
- Performance metrics: Latency, throughput, utilization, SLAs
- A systems approach to designing for performance
- Measurement-driven approach to build high-performance systems
  - Identifying bottlenecks (time-based profiling)
  - Automated performance profiling tools (Linux perf and flamegraphs)
- Design hints for performance:
  - Resource splitting
  - Caching
  - Compute in the background
  - Batch processing
  - Parallelism
Part IV: Pattern implementation
- Model-view-controller (MVC) pattern

L06 System Design and Implementation III

Part I: Concurrency (or Scale Up!)
- Why Concurrency?
  - Single-node parallelism: multicores, accelerators, smart devices (SSDs/NICs)
- Process vs threads
- Thread scheduling
  - Cooperative vs preemptive schedulers
  - Scheduling policies: round robin, fairness-based, priority-based, earliest-deadline first
- Parallel programming
  - Managing threads: thread spawning, thread pools
  - Communication mechanisms and critical sections
  - Synchronization primitives: Semaphores, mutexes, barriers, readers-writer locks
  - Lock-free data structures
- Problems with threads
  - Race Conditions
  - Deadlocks, livelocks
  - Starvation
- Classical synchronization problems
  - Dining philosophers
  - Sleeping barber
- Parallel programming patterns
  - Fork-join
  - Data-parallel programming pattern w/ barriers
  - Synchronous vs asynchronous
- Programming accelerators
  - Programming model
  - GPU architecture
  - FGPA architecture
Part II: Scalability (or Scale Out!)
- Why scalability? (Limitations of single-node scaling up!)
- Challenges of scalability
- Scalable architectures
  - Primary-worker architectures
  - Differentiation between the control and data paths
- Scaling stateless applications
  - load balancers
- Scaling stateful applications
  - Sharding
  - Replication
  - Consistent hashing
  - Secondary indexes
- Scaling computations
  - Data-parallel programming model (MapReduce)
Part III: Pattern implementation
- Adapter pattern
- Observer pattern
- Strategy pattern

L07 Software Testing

Part I:
- Testing overview
- Unit testing, integration testing, system testing
- Unite testing:
  - Apply unit testing with JUnit
Part II: Mock testing
- Test doubles
- Explain model-based and object-oriented testing
Part III: Automated large-scale testing
- Fuzzing
  - Fuzzing w/ AFL
- Symbolic execution
  - KLEE, S2E, Angr
System crashing for resiliency
- - Chaos Monkey at Netflix

L08 Program analysis

Part I: Faults and failures in software
- Terminology and impact
Part II: Program analysis trade-offs
- Soundness, completeness, static vs. dynamic analysis
Part III: Static analysis tools
- Compiler warnings/errors
- Infer
- Clang Analyzer and Clang Tidy
- Spotbugs
Part IV: Brief introduction to C
Part V: Dynamic analysis tools
- Undefined behavior
- Memory safety issues
- Dynamic binary instrumentation
- Compiler-assisted instrumentation (LLVM)
- Valgrind
- Sanitizers (AddressSanitizer, ThreadSanitizer, MemorySanitizer, UBSan)

L09 Software management and deployment

Part I: Source code management
- Source code management
- Version control
- Branch management
- Centralized (piper) vs decentralized (git) source code management
Part II: Build systems
- Why a build system?
- Task-based build systems
- Artifact-based build systems
- Distributed builds
- Dependency management
- Hermeticity
Part III: Release management
- Release planning
- Software versioning
- Software upgrades
Part IV: Deployment automation
- Continous integration
- Continous delivery
- Continous deployment
- Continous testing/fuzzing w/ OSS Fuzz
Part V: Cloud systems monitoring
- Why system monitoring?
- Prometheus architecture
- Metrics types
- Visualizing metrics w/ Graphana
- Altering mechanisms
- An example: Instrumenting a simple HTTP server

L10 Project management + Software quality + Exam Preparation

Part I: Software quality
- Software quality management
- Reviewing
- // Comments
- Code refactoring
- Trustworthy software systems
  - Formal verification
  - Code compliance
Part II: Project management
- Project management
- Work breakdown structure
- Team organization
- Communication mechanisms
Part III: Exam preparation
- Exam format
- Q&A

Teaching and Learning Methods

By means of slide presentations with animations, the interactive lecture introduces the basic concepts and methods of software engineering and explains them using examples. Small exercises, e.g. quizzes, text exercises, and programming tasks, with individual feedback help students to identify whether they have understood the basic concepts and methods.

Accompanying tutorials deepen the understanding of the concepts explained in the lecture by means of suitable team exercises and show the application of the different methods with the help of manageable problems in the different phases of software engineering. Homework enables students to deepen their knowledge in self-study. The presentation of the own solution in the accompanying tutorials improves communication skills, which are essential in software engineering. Individual feedback on homework allows students to measure learning progress and improve their skills.

Media

Lecture with digital slides, live stream recordings, online exercises (programming, quiz, text) with individual feedback, a communication platform for the exchange between instructors, tutors, and students.