Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Preface

SovScaDesDisMaLOps := Sovereign Scalable Data Engineering Sciences and Distributed Machine Learning Operations

SovScaDesDisMaLOps Workshop for the Wallenberg AI, Autonomous Systems and Software Program (WASP-AI) Community Building Summer School in 2025.

This book contains learning material, divided up into slide decks. Each deck constitute a lesson. Several lessons combine to form a module, and a series of modules constitute a learning workshop.

This is the book version of the material. You can also see the lessons as slide decks and as pure markdown here:

Credits

The development of this course is supported by the Wallenberg AI, Autonomous Systems and Software Program funded by Knut and Alice Wallenberg Foundation.

The course materials are open sourced as a contribution to the community of learners. The ideas and concepts as well as working codes will be made available here for self-study and group-study for anyone who is interested in SovScaDesDisMaLOps.

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. We encourage the use of this material, under the terms of the above license, in the production and/or delivery of commercial or open-source learning programmes.

Copyright (c) VakeWorks, 2025

VakeWorksTM_black logo
VakeWorksTM_white logo

Definition

Definition of SovScaDesDisMaLOps

  • SovScaDesDisMaLOps := Sovereign Scalable Data Engineering Sciences and Distributed Machine Learning Operations

  • DesDisMaL := Data Engineering Sciences and Distributed Machine Learning

    • matematics + statistics + computing + micro-economics + domain-expertise

  • SovSca_Ops := Sovereign and Scalable (Business and Digital) Operations

    • law + entrepreneurship + operations
    • sovereign = being independent and free from the control of another
    • scalable = being able to grown on demand

SovScaDesDisMaLOps
SovScaDesDisMaLOps := Sovereign Scalable Data Engineering Sciences and Distributed Machine Learning OperationsDesData Engineering SciencesSovScaDesDisMaLOpsSovereign Scalable Data Engineering Sciences and Distributed Machine Learning OperationsDes->SovScaDesDisMaLOpsDisMLDistributed Machine LearningDisML->DesMLOpsMLOps = Machine Learning, DevOps, Data Engineering & SecurityMLOps->DesMLOps->SovScaDesDisMaLOpsLawSovLaws of Capitalism, Contracts, IP, Jurisdiction, SovereigntyEntrpEntrepreneurship,Business ScalabilityLawSov->EntrpLawSov->SovScaDesDisMaLOpsConThemicroeconomics statistical contract theoryConThe->DesConThe->LawSovMATHSMathematicsMATHS->DesSTATSStatisticsSTATS->DesCOMPTComputing and Software EngineeringCOMPT->DesDOMEXDomain Expertise (in Science, Engineering, Art, Humanities, etc.)DOMEX->DesEntrp->SovScaDesDisMaLOps

Reference Books

RefBooks

See LI post for context.

Overview

THM: Theoretical Modules -- Overview

  1. [1 hour] Laws of Capitalism, Statistical Contract Theory, Intellectual Property Laws in Sweden, Corporate Jurisdictions and Sovereignty
  2. [1 hour] Abstract Machine Models, Work-Depth Model, Brent's Theorem, and Analysis of the Fastest Sorting Algorithm in Distributed Parallel Random Access Machine (DPRAM) model

PRM: Practical Modules -- Overview

  1. [1 hour] Scaling Python with Ray in a Hurry (requirements: git or github account; docker desktop; up-to-date and secure laptop)
  2. [1 hour] Deploying for Sovereign Operations in Scalable Data Engineering Sciences and Distributed Machine Learning Spaces.

Prepare Your Laptop

  • You will be learning the practical modules on your own laptops.
  • Prepare your laptop with the following three tasks well before the workshop.

Task 1: Update System

Task 2: Install Docker

Task 3: Check Docker Works

  • Check that your docker installation was successful

    • Open your CLI terminal and start a container by running the command:

      docker run hello-world
      • You should receive a Hello from Docker! in the output.

    • Run welcome-to-docker as daemon mapping port 80 to 8080.

      docker run -d -p 8080:80 docker/welcome-to-docker
      • Visit http://localhost:8080 in your browser to access this container and see a welcome message.
      • Identify and Stop the running welcome-to-docker container.

  • [OPTIONAL] You can checkout command-line and docker pages we will be going through.

Laws of Capitalism

Quick Tour of Laws of Capitalism

  • Katharina Pistor breaks down the history, process, institutions, and participants involved in the legal coding of capital.
  • She shows us how private actors have harnessed social resources to accumulate wealth, generating not only economic inequality, but inequality in law.
    • Enabling them to opt out of jurisdictions, restrict governmental policy, and erode democracy.
  • The laws of capitalism have elevated the interests of the few above that of the many, but we can rewrite the code and restore balance to society.

Coding Land & Ideas

  • how the law selectively "codes" certain assets, endowing them with the capacity to protect and produce private wealth
    • how land became legally coded as property during the enclosure movement in England
    • attempts to code traditional indigenous land use rights in Belize.
  • explains how even ideas, which are not natural property, can be made an entire new class of property according to the law
    • institutionalization of copyright and patent law
    • coding intellectual property; eg. patent human genes

Coding Debt

How & why law codes debt

  • how the law has accompanied the evolution of negotiable debt instruments, from the
    • simple IOU, to
    • bills of exchange in Medieval Genoa, to
    • securitized assets in the modern day and on to derivatives
  • examines the securitization of mortgage debt in the modern United States in the lead-up to the financial crisis of 2008
  • the legal code has been essential in the transformation of humble IOUs
    • into elaborate financial contracts, and
    • the creation of an entire finance industry

Lawyers, Firms & Corporate Entities

Corporations to create wealth 1600s+

  • how the law has accompanied the evolution of private business enterprises,
    • from their early roots as simple partnership agreements of temporary duration,
    • to colonial joint-stock companies in the 1600's with state charters granting them the legal status of corporations (or legal persons),
    • and the eventual introduction of general incorporation statutes in the 1800's simplifying the creation of corporations.

Corporations to create wealth today

  • by the 1900's the law enabled the corporation to become the dominant form of business organization under capitalism
  • case study of Lehman Brothers: how its organization as a holding company allowed shareholders to profit, while leaving debt-strapped subsidiaries to flounder.
    • Lehman harnessed the corporate structure created by law to build a house of cards.

The Tool Kit

  • the four essential attributes of capital:
    • priority,
    • durability,
    • universality, and
    • convertibility.
  • these ensure that an asset generates wealth for its owner, i.e. becomes capital

How to legally code assets as capital?

  • through the legal institutions constructing certain bodies of law:
    • property law,
    • collateral law,
    • trust law,
    • corporate law,
    • bankruptcy law and
    • contract law.
  • private owners of capital can harness the centralized means of coercion (like litigating in State courts) to make their rights effective
  • property rights can also be pursued in a more decentralized fashion, with parties picking the law and the forum.
  • legal claims are pursued at various venues, but also where law is constructed
  • law is not always handed down from above, but often emerges from below, sometimes outside the courts
  • law and legal institutions are not static, but evolve and expand by an incremental process, usually driven by interested parties.
  • it need not be by legislation - they can change the meaning of a law by renegotiating its interpretation and application.

The Codemaster

Private lawyers in coding capital

  • Private Attorneys are the masters of the code
  • Despite the fact that the State and juridical precedent establishes the law, it is private attorneys who find the flaws, loopholes, or inaccuracies in the legal code
  • history of private lawyers in Prussia, France, and the American legal system
  • private lawyers shape and mold the legal code over time
  • vast amounts of wealth accumulates in private law firms today
    • e.g. relationship between the UK’s Magic Circle of Firms and Russian Oligarchs.

A Code for the Globe

How law codes international capital?

  • Laws are national, but international capital transcends borders.
  • the domestic law of two hegemons:
    • the United Kingdom and
    • the United States
  • have become the default legal systems used internationally
  • globalization has been made possible by using these two dominant jurisdictions to code a global system for trans-national commerce, finance and corporations

Trans-national laws of capitalism

  • individual countries may have ceded too much for international capital
  • bilateral investment treaties lead to tension between democratic sovereignty and globalization
  • desirable domestic regulations and protections sought by the democratic process can be hampered by the threat of expensive lawsuits from foreign investors
  • freedom of capital movement across borders brings along an alien legal jurisdiction, imposing restrictions on local governments, elevating the private rights of capital above the public interest, and eroding democracy

Transforming the Code of Capital

how the code of capital is used?

  • the legal system is a social resource, but it has been harnessed by private actors to create and accumulate immense private wealth.
  • This has not only produced economic inequality today, but also inequality before the law.
  • Private actors can opt out of jurisdictions, restrict the policy space of governments, and erode democracy.
  • we can rewrite the code to make it fairer.
    • it can be an incremental process
    • including rolling back some of the privileges and exemptions acquired by capital over the years,
    • setting out jurisdictional boundaries to ensure that the same rules apply to everyone.
    • rules can also be adjusted to ensure those who benefit internalize their costs

conscientious citizens of democracies

  • the burden need not always fall on the government
    • tools and institutions can be created to allow private actors to monitor capital more effectively
  • the legal code can be changed to make the law fairer and
  • allow governments to regain some policy space,
    • to address the needs and implement the protections desired by the people through the democratic process
    • however, sovereignty of operations is a pre-requisite to allow the government of your jurisdiction to be able to act.

Any corporation can choose its bylaws!

...shows how 12 CEOs ... uniquely rejected narrow yardsticks of shareholder value by leading companies to larger models of prosperity and justice... This select group of mavericks and iconoclasts — which includes The Body Shop’s Anita Roddick, Patagonia’s Yvon Chouinard, Vanguard’s John Bogle and Busboys and Poets’ Andy Shallal —give us, Nader writes, “a sense of what might have been and what still could be if business were rigorously framed as a process that was not only about making money and selling things but improving our social and natural world.

You can be a rebellious CEO!

  • Given the urgency of climate change and increasing wealth ineqalities:
    • there is no need to wait for the laws of capitalism to be revised.
  • We will look at considerations for starting a corporation in Sweden
    • specifically at legal codes favouring PhD students in Sweden
  • Sweden is ideal for innovation
    • Sweden ranks 2nd among the 133 economies featured in the Global Innovation Index 2024
    • The land of unicorns: How did Sweden end up with the second largest concentration of billion dollar companies per capita in the world? Professor Robin Teigland explains how a small country nestled away in the Nordics developed its magic touch.

  • Minimal Exercise: WATCH Laws of Capitalism Playlist [140 minutes] *

    The laws of capitalism have elevated the interests of the few above that of the many, but we can rewrite the code and restore balance to society. In this series, Professor Katharina Pistor (Columbia Law School) breaks down the history, process, institutions, and participants involved in the legal coding of capital. She shows us how private actors have harnessed social resources to accumulate wealth, generating not only economic inequality, but inequality in law. Enabling them to opt out of jurisdictions, restrict governmental policy, and erode democracy. Learn more at http://lawsofcapitalism.org/

  • Optional Deeper Dives:

    • READ The Code of Capital: How the Law Creates Wealth and Inequality, Katharina Pistor, Princeton University Press, 2019, ISBN: 9780691178974.
      • Chapters: Empire of Law, Coding Land, Coding Legal Persons, Minting Debt, Enclosing Nature's Code, A Code for the Globe, The Masters of the Code, A New Code?, Capital Rules by Law.
    • READ The Law of Capitalism and How To Transform It, Katharina Pistor, coming in September 2025. *

      A fascinating study of the legal underpinnings of capitalism, why the system must be transformed, and what we can do about it. While capitalism has been conventionally described as an economic system, it is actually a deeply entrenched legal regime. Law provides the material for coding simple objects, promises and ideas as capital assets. It also provides the means for avoiding legal constraints that societies have frequently imposed on capitalism. Often lauded for creating levels of wealth unprecedented in human history, capitalism is also largely responsible for the two greatest problems now confronting humanity: the erosion of social and political cohesion that undermines democratic self-governance on the one hand, and the threats that emanate from climate change on the other. By exploring the ways that Western legal systems empower individuals to advance their interests against society with the help of the law, Katharina Pistor reveals how capitalism is an unsustainable system designed to foster inequity. She offers ideas for rethinking how the transformation of the law and the economy can help us create a more just system—before it’s too late.

Intellectual Property Laws in Sweden

Professor's Privilege Law (lärarundantaget):

In 1949, a new principle was introduced in the Swedish Patent Act: “Teachers at universities, colleges or other establishments that belong to the system of education should not be treated as employees in the scope of this law” (SFS 1949:345: §1). In practice, the formulation meant that academic scholars were not subject to a new patent law which gave employers the intellectual property rights to inventions made by employees. Or in other words: a patent generated by academic research belonged to the scientist, not to the university. Today – the legislation is still in place – this exception for university scholars is known as the “professor’s privilege” or “teacher's exemption” and lärarundantaget in Swedish ...

Starting a Business in Sweden

Corporate Jurisdictions and Business Sovereignty

  • Corporate entities in Sweden?
    • Sole tradership (Enskild firma), Limited Liability Company (Aktiebolag) and others
  • Structre of Aktiebolag (AB)
    • Board of Directors ("responsible for operations"), List of Shareholders ("fractional owners")
  • Resources:

Statistical Contract Theory

Basics of the Principal-Agent Model under Information Asymmetry.

  • Main Assumptions

    • An agent and a principal enter into a contract with their own self-interests
    • Coercive powers of a State exist (Courts can enforce contracts)
    • There is information asymmetry:
      • agent won't share crucial private information with principal or vice versa

  • It's a take-it-or-leave-it contract from principal to agent

  • Example 1: Airline industry

    • Principal is airline; agent is passensger;
    • information hidden from principal: the maximum price an agent is willing to pay per trip
    • Story: how was the airline industry rescued by contract theory half a century ago?

  • Example 2: Streaming Music Indusrty - United Masters

    • Principal-S is a streaming service provider, eg.spotify, disney music, YouTube Music, etc.
    • Agent-P is a musician producing music
    • Agent-C is a listener consuming music
    • Traditional contract: Agent-P <- Principal-S -> Agent-C
    • United Masters as Principal-UM have a different contract model:
      • Agent-P <- Principal-UM -> Agent-S, where Agent-S is a streaming provider

  • Example 3: Food Industry - Radically Transparent Food Networks at www.shop.vake.works

    • Food reaches from producer or farmer to eater through a sequence of pairwise contracts between different types of agent-principal pairs.
    • Consider charcuterie made of beef, salt and pepper, then the following pairs may exist before it reaches the eater:
      • producer-processor (eg. slaughterer, butcher, packager, cold-chain provider)
      • processor-transporter, transporter-distributor, distributor-exporter, exporter-importer, importer-distributor, distributor-retailer
      • retailer-eater

  • Reference Readings:

Analysis of Algorithms & Operations

DPRAMJAM := Distributed Parallel Random Access Machine Jurisdiction-Aware Model

[1 hour]

  • Abstract Machine Model (AMM)
    • SRAM: Sequential Random Access Machine Model (older computers)
    • PRAM: Parallel Random Access Machine Model (laptops and smarart phones)
    • DPRAM: Distributed Parallel Random Access Machine Model (cluster of computers or cloud)
  • Asymptotic analysis of efficiency of algorithms under different AMMs
    • SRAM PRAM and DPRAM models with 1, 2, ..., p, p+1, ... processors and a memory model
    • Work-Depth Model and Brent's Theorem
    • Example + : Addition of n numbers in SRAM, PRAM and DPRAM models
      • we will revisit this example implemented using ray.io in a memory model
    • Fastest Sorting Algorithm in DPRAM model = d distributed PRAM models
  • Legal analysis of sovereignty of operations under Jurisdiction-Aware Model (JAM) extend AMMs beyond analysis of algorithms
    • JAM-local-Sverige is our default model when we work in Sweden (Sverige) with sovereign infrasturcture in Sweden
      • Example: own laptops (default for this workshop), workstations, or own on-premise clouds (NuC cluster) under sovereignty of Sverige
    • many clouds in a sky: "sky computing" means your service is agnostic to a set of clouds - Example: databricks offers sky computing across some clouds: AWS, GCP and Azure
    • JAM-sky-State extends DPRAM by specifying the Nation State of the cloud-provider's HQ, i.e., the State that can enforce contracts and regulate each cloud in the sky.
    • Examples of JAM-sky-State, a set of cloud providers under the jurisdiction of a State:
      • JAM-sky-USA includes AWS, GCP, Azure, etc.
      • JAM-sky-China includes Alibaba, Tencent, Huawei, etc.
      • JAM-sky-EU includes scaleway, OVHcloud, Exoscale, Elastx, etc.
      • JAM-sky-Sverige includes Elastx, bahnhof, etc.
    • DPRAMJAM-sky-State = Distributed Parallel Random Access Machine Jurisdiction-Aware Model is the "Math+Law" model required for sovereign operations in the sky of a said State
      • i.e., to enforce contracts or prohibit their violations or regulate possible contracts through coercive powers of the said State and not by another State.
  • Break-time! ... AoAO... AoAO ahhhaaaa!

Command-Line Interface

In the beginning, was the command-line...

$ toilet command-line
                                               #   ""#      "                 
  mmm    mmm   mmmmm  mmmmm   mmm   m mm    mmm#     #    mmm    m mm    mmm  
 #"  "  #" "#  # # #  # # #  "   #  #"  #  #" "#     #      #    #"  #  #"  # 
 #      #   #  # # #  # # #  m"""#  #   #  #   # """ #      #    #   #  #"""" 
 "#mm"  "#m#"  # # #  # # #  "mm"#  #   #  "#m##     "mm  mm#mm  #   #  "#mm"
  • A command-line interface (CLI) is a means of interacting with software via commands
    • each formatted as a line of text
  • CLIs emerged in the mid-1960s, on computer terminals, as an interactive alternative to punched cards
  • Alternatives to a CLI include a GUI (Graphical User Interface) but GUI is not conducive to automating programs
  • CLI and GUI are interfaces around the same thing and can be used interchangeably
    • except that CLI is much faster and reliable when managing a cluster or server farm
  • Resources for deeper dives:

Let's Try Together Now!

Let's try the following commands on our command-line interface (CLI), also known as shell or terminal now.

The commands are adapted from Wallace Preston's Ultimate Terminal Cheatsheet - Your Key to Command Line Productivity.

Bookmark https://cheatsheetindex.com/ for many more tools.

Master the Terminal and Rule the Command Line Kingdom!

Welcome to the Ultimate Terminal Cheatsheet! We know the command line can be intimidating, like trying to navigate a maze blindfolded. But fear not, for this cheatsheet will be your trusty guide, like a GPS for your terminal. From manipulating files to networking to version control, we've got you covered. Let's power up our keyboards and get ready to conquer the command line!

Basic Terminal Commands

  • Command line
    • See where I am pwd
    • list files ls
    • list hidden files ls -a
    • change directory cd <directory-name>
    • Open whole directory in VS Code code .
    • shorthand for current and previous folder
      • ls . lists current folder
      • ls .. lists previous folder
      • cd .. moves up a folder
    • home vs root
      • ~ is short for home
      • cd ~ brings you home
      • / is short for root
      • cd / brings to you root
    • absolute vs relative paths
      • Absolute: touch /folder-name/file-name.txt goes to the root folder, into folder-name, makes a file inside of that folder
      • Relative: touch ./folder-name/file-name.txt goes into folder-name from current level, makes a file inside of that folder
      • touch ~/folder-name/file-name.txt goes into your home directory (usually looks like /home/username)
    • Files
      • create files touch file-name.txt
      • remove a file rm file-name.txt
      • rename a file mv file-name.txt new-file-name.txt
      • move a file mv this.txt target-folder/
    • Folders
      • create folders mkdir folder-name
      • remove a folder rm folder-name -R
      • rename a folder mv folder-name new-folder-name
      • move a folder mv folder-name ../../target-folder
    • grep
      • search recursively for a pattern in a directory: grep -r <pattern> <directory>
      • search for a pattern in a file: grep <pattern> <file>
      • search for a pattern in multiple files: grep <pattern> <file1> <file2> <file3>
      • ignore case when searching: grep -i <pattern> <file>
      • count the number of occurrences of a pattern: grep -c <pattern> <file>
      • print the line number for each match: grep -n <pattern> <file>
      • print the lines before or after the matching line: grep -A <num_lines> <pattern> <file> or grep -B <num_lines> <pattern> <file>
      • print the matching line and a few lines before and after it: grep -C <num_lines> <pattern> <file>

ports/processes

  • Show what is running on a port lsof -i:<port-number>
  • Kill a process kill <process-id>
  • Example of above two steps
$ lsof -i:3000
COMMAND   PID    USER   FD   TYPE            DEVICE SIZE/OFF NODE NAME
node     78070 myuser   24u  IPv6 0xdc9d8g07c355e5f      0t0  TCP *:hbci (LISTEN)

$ kill 78070
  • Show list of all node processes ps -ef | grep node
  • Kill all node processes pkill -f node

VIM

  • Insert mode: i
  • Normal mode: Esc
  • Write changes to file: :w
  • Quit VIM: :q
  • Save changes and quit VIM: :wq
  • Undo last change: u
  • Redo last undone change: Ctrl + r

Git

  • Cloning and remotes
    • Clone git clone <url>
    • Check where cloning from and committing to git remote -v
    • Add a remote repository git remote add <remote-name> https://remote-url/repo-name.git
    • Change the origin git remote set-url origin <new-git-repo-url>
  • Committing
    • See status of staging area etc git status
    • Add all files in the repo to staging area git add -A
    • Add just one file to the staging area git add <filename>
    • Commit git commit -m "my-message"
    • Push git push
  • Branching
    • Check out to a new branch git checkout -b <branchname>
    • Check out to an existing branch git checkout <branchname>
    • Delete a branch (cannot be the branch you're on!) git branch -D <branchname>
    • Merge another branch into current branch
      1. Move to the branch you want to merge FROM git checkout dev
      2. Pull down most recent remote version git pull
      3. Move to the branch you want to merge TO git checkout feature-branch-#25
      4. Merge the FROM branch (dev in this case) into the current branch git merge dev
  • Logs
    • See log of activity (commits) git log
    • See log with limited info (easier to read) git log --oneline
  • Configs
    • Change git email locally: git config user.name "Marley McTesterson"
    • Change git email locally: git config user.email example@email.com
    • See git repo config settings (i.e. email username etc): git config --list
  • Diffs
    • See what is changed currently compared to the most recent commit git diff
    • See what has changed in the current working tree vs a specific branch git diff <branch-name> --
    • Show the difference between two branches: git diff <branch1> <branch2>
  • Removing/rebasing changes - Intermediate to Advanced
    • Auto-stash changes before switching branches: git checkout <branch> --autostash
    • Remove uncommitted changes (but save them for later, just in case) git stash
    • Remove last commit git reset --hard HEAD^
    • Squash the last 3 commits into one single commit git rebase -i HEAD~3
      • Then in the editor, for any commits you want to keep leave pick. For anything you want to "squash" and not include that as a commit, change pick to squash.
    • Squash everything after git rebase -i e25340b where e25340b is the SHA1
      • Repeat next step from above to pick/squash specific commits.

cURL

  • Methods
    • GET request: curl https://www.google.com/
    • POST request: curl http://localhost:3000/myroute -H "Content-Type: application/json" -X POST -d '{"key1": "val1", "key2": "val2"}' (-H is a header, in this case setting our content type as json. -X is setting the method, post in this case. -d is the request body in JSON format.)
    • PATCH request: curl http://localhost:3000/myroute -X PATCH -H 'Content-Type: application/json' -d '{"key1": "val1", "key2": "val2"}'
    • DELETE request: curl http://localhost:3000/myrouote -X DELETE
  • Setting Authorization header: curl http://localhost:3000/myroute -H 'Authorization: Bearer <myreallylongtoken>'
  • Basic Auth: Authenticate with a username and password while accessing a URL: curl -u <username>:<password> <url>
  • Send a POST request with data from a file: curl -X POST -d @<filename> <url>
  • Fetch only the headers of a URL: curl -I <url>
  • Download the contents of a URL and save it to a file: curl -o <output_file> <url>

Serverless Framework

  • Create a new serverless project: serverless create --template <template-name>
  • Deploy a serverless project: serverless deploy
  • Remove a deployed serverless project: serverless remove

Docker

  • List all running containers: docker ps
  • Stop a container: docker stop <container-id>
  • Remove a container: docker rm <container-id>
  • List all available images: docker images
  • Remove an image: docker rmi <image-id>

macOS

  • Show hidden files in Finder: Shift + Command + .
  • Open a file or folder in Finder from Terminal: open <file/folder>
  • Show all running processes: ps aux
  • Kill a process: kill <pid>
  • Restart the macOS Dock: killall Dock

Linux

  • Update package manager: sudo apt-get update
  • Install a package: sudo apt-get install <package-name>
  • Remove a package: sudo apt-get remove <package-name>
  • Show all running processes: ps aux
  • Kill a process: kill <pid>

Windows

  • We can fall back on Linux in this case as we will be running docker.

Docker

What is Docker?

Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allows you to run many containers simultaneously on a given host. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host. You can easily share containers while you work, and be sure that everyone you share with gets the same container that works in the same way.

Download CLI Cheat Sheet that we will go through next.

DOCKER INSTALLATION

Docker Desktop is available for Mac, Linux and Windows

View example projects that use Docker

Check out our docs for information on using Docker

Alternatively, On Linux Install Docker Engine

DOCKER IMAGES

Docker images are a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

 # Build an Image from a Dockerfile without the cache
docker build -t <image_name> . –no-cache

 # List local images
docker images

 # Delete an Image
docker rmi <image_name>

 # Remove all unused images
docker image prune

DOCKER HUB

Docker Hub is a service provided by Docker for finding and sharing container images with your team. Learn more and find images at https://hub.docker.com

 # Login into Docker
docker login -u <username>

 # Publish an image to Docker Hub
docker push <username>/<image_name>

 # Search Hub for an image
docker search <image_name>

 # Pull an image from a Docker Hub
docker pull <image_name>

DOCKER GENERAL COMMANDS

 # Start the docker daemon
docker -d
 # Get help with Docker. Can also use –help on all subcommands
docker --help
 # Display system-wide information
docker info

DOCKER CONTAINERS

A container is a runtime instance of a docker image. A container will always run the same, regardless of the infrastructure. Containers isolate software from its environment and ensure that it works uniformly despite differences for instance between development and staging

 # Create and run a container from an image, with a custom name:
docker run --name <container_name> <image_name>

 # Run a container with and publish a container’s port(s) to the host.
docker run -p <host_port>:<container_port> <image_name>

 # Run a container in the background
docker run -d <image_name>

 # Start or stop an existing container:
docker start|stop <container_name> (or <container-id>)

 # Remove a stopped container:
docker rm <container_name>

 # Open a shell inside a running container:
docker exec -it <container_name> sh

 # Fetch and follow the logs of a container:
docker logs -f <container_name>

 # To inspect a running container:
docker inspect <container_name> (or <container_id>)

 # To list currently running containers:
docker ps

 # List all docker containers (running and stopped):
docker ps --all

 # View resource usage stats
docker container stats

Try Some Docker Commands

  • Open your CLI terminal and start a container by running the command:
docker run hello-world

  • You should receive a Hello from Docker! in the output.

  • Run welcome-to-docker as daemon mapping port 80 to 8080.

docker run -d -p 8080:80 docker/welcome-to-docker
  • Visit http://localhost:8080 in your browser to access this container.
  • Identify and Stop the running welcome-to-docker container.
 # view available docker images
docker image ls

 # view running docker container processes
docker ps

 # stop a running container by its CONTAINER ID or NAMES
docker stop <CONTAINER ID>

Ray

  • Ray core walkthrough
    • functions -> tasks
    • classes -> actors
    • basics of ray design
  • Reinforcement learning from ray core scratch
  • pointers to libraries and ecosystem

Sovereign Operations

We need sovereign deployment of our algorithms and operations to possibly provide SovScaDesDisMaLOps-based services that create "value" to help build a better world through an Algorithms-Machines-Operations-Lives-Planet perspective.

  • Ideally we want to learn tools that can ease JAM-sky-* deployment of our ray services in an arbitrary State's jurisdiction by learning from DPRAMJAM-local-Sverige on our laptops, to be able to deploy to various DPRAMJAM-sky-STATEs with little extra effort.
  • To do this, beyond command-line, docker and local ray clusters, we need to build further with:
    • kubernetes
    • terraform.io
    • ...

You can study these on your own from the provided references or take the WASP PhD course ScaDaMaLe with me in 2026 fall.