Linux/Unix-like Environments

Unix-like Environments (sometimes written “*nix environments”) refer to any of a family of operating systems that were either derived or based on the early PDP and Unix operating systems of the 1960s and 1970s. Today when people refer to Unix-like Environments, they typically mean a macOS or Linux machine.

Nonetheless, there are many systems that share similarities with the two:

Other operating systems developed in parallel: such as the Windows operating systems that was based on DOS (another early operating system, and an acronym for “Disk Operating System”). Understanding the details of an operating system may be important for specific tasks—if we were developing video games and planning for them to be played on Windows, we would want to better understand details of the Windows platform.

However, Unix-like environments—and Linux environments specifically—are prolific across a range of computing problems. When a user navigates to a web page, their browser (e.g. Firefox, Chrome, Edge) is almost certainly communicating with several other computers running some flavor of Linux.

“The Cloud” sounds like a deific object, but in reality the cloud is a room full of machines running virtual machines inside of them: launching into existence just long enough to accomplish some task before exporting some data and disappearing entirely. In other words: “The Cloud” is referring to to a series of Unix-like environments.

We’ll focus on three key skills in this section:

1. Familiarity with File Systems. A File System organizes content into files and folders (which we’ll start referring to as “directories” soon).

2. Knowing enough about Terminals and Shells to move around, launch programs, and be productive. Most modern operating systems have a graphical shell called a “Desktop” with buttons—but an interface supporting text input and text output is frequently the quickest (or only) means of accomplishing computing tasks.

3. Having enough knowledge of Process Management to launch processes, wait on them to complete, debug them, or end them if necessary.

We will practice working in Unix-like environments here. macOS users already have this environment. Windows users should use the WSL (Windows Subsystem for Linux) to maintain a seemless experience with the material.¹

Perfectly Spherical Operating Systems

An operating system is a program that manages hardware or computer resources.

Mainstream operating systems are built around a hierarchical tree structure created out of directories (folders) and the files inside of the directories. A file stories data, and directories might be used to group related data together in some logical way.

Directories

There exists a root directory. That root directory contains other directories. These two facts create a parent-child relationship between directories.

Every child can have children of its own: but every child has one and only one parent.² This means that the parent-child relationships could (in principle) extend like nesting dolls infinitely far down into folders-inside-of-folders-inside-of-folders:

Assume everything above is in its place, and give a special name using the tilde ~ to represent the home directory. Everything from earlier is still true, but we’ve constrained the universe where only a specific part of it is relevant. As Unix users, our actions are almost always relative to our home directory: the lower levels do exist, meaning the levels closest to “root” or /, but they’re safe to ignore during most day-to-day activities:

The Same Person with Many Faces

On each operating system that you’re likely to encounter: there exists some concept of “root” and “home.” They look slightly different though:

macOS	/Users/hayesall/
Linux	/home/hayesall/
Windows	C:\Users\hayesall\

Each has slightly different behavior, but most programs will assume a home directory ~ exists for the user, and that the home directory and (by transitivity) everything inside of it belongs to the user.

Follow Along with the Instructor

Practice with the instructor. Not an exact replacement for the written directions below.

• Become comfortable with the command line interface and the structure of Linux/Unix file systems

Files and Directories

Open your Terminal (Mac) or WSL (Windows) to follow along.

Let’s create a new directory called SampleDir in the user’s home directory, and put another directory called 1 inside of SampleDir.

• Everytime you see ~ a tilde, it means the user’s home directory.
• A forward slash / indicates a directory.

mkdir ~/SampleDir
mkdir ~/SampleDir/1

This creates the following structue; a directory called “SampleDir” containing another directory called “1”:

SampleDir
└── 1

Now let’s put files in those directories.

touch ~/SampleDir/2
touch ~/SampleDir/3
touch ~/SampleDir/1/1

The touch command takes a path including a file and creates a file. In this output, the numbers 1, 2 and 3 are folders (directories), but the nested “1” is a file:

SampleDir
├── 1
│  └── 1
├── 2
└── 3

You now know how to create both directories and files!

Some basic Unix commands

It might help to think of the next few commands as controlling a cursor.

In a graphical shell (how you’ve probably used computers previously), one can use the mouse to move a cursor on a screen, and click/double-click to move into folders. Typically: there is also some text providing feedback for where one is.

Where are you?

pwd » Print Working Directory

For example: pwd for Alexander shows /home/hayesall

What is in here, or there?

ls » List files in the current directory

ls is also a program which takes a file path as an argument. For example: ls /home/hayesall/ will show the files in that directory.

Linux distributions often avoid polluting the user’s home directory with unnecessary files and folders. Therefore, if you’re on the Windows Subsystem for Linux, you will not see anything by default:

$ ls /home/hayesall

On a Mac, you’re likely to see familiar folders such as “Desktop”, “Documents”, or “Downloads”.

Dollar signs? $ / Percent signs? %

In technical writing, authors often use a dollar sign $ or percent sign % to communicate when operations are done in a terminal or text shell. The $ is a common prompt token indicating that the shell is ready for the next command. The default on Linux typically looks like: hayesall@hayesall:~$, or like ebigalee@sice ~ % on macOS.

The takeaway: do not copy the dollar sign when you’re following along, you’ll probably get something like: $: command not found.

There are files in the home directory, but they are hidden files that start with a period .. These dotfiles are often used for configuration in Unix-like environments.

To view hidden files, you can add an option called a flag, to your command. A flag modifies the behavior of a command, giving you access to additional functionality when needed. For example:

ls -a » List files in current directory, also showing hidden files

$ ls -a ~
.  ..  .bash_history  .bash_logout  .bashrc  .local  .profile  .viminfo

ls -l » List files in current directory, in long listing mode

$ ls -l
total 0

The long listing for files gives information on resource information (d for directory, or - for file) file permissions (is it readable r, writable w, or executable e?), ownership, file size (in bytes), and the last modified timestamp.

But since our files are hidden, we need to use both the -a and the -l flags:

$ ls -a -l
total 36
drwxr-x--- 3 hayesall hayesall 4096 Jun 13 10:01 .
drwxr-xr-x 5 root     root     4096 May 15 12:38 ..
-rw------- 1 hayesall hayesall 1223 Jun 12 15:16 .bash_history
-rw-r--r-- 1 hayesall hayesall  220 May 15 12:38 .bash_logout
-rw-r--r-- 1 hayesall hayesall 3771 May 15 12:38 .bashrc
-rw------- 1 hayesall hayesall   20 Jun 12 14:03 .lesshst
drwxrwxr-x 3 hayesall hayesall 4096 May 15 12:39 .local
-rw-r--r-- 1 hayesall hayesall  807 May 15 12:38 .profile
-rw------- 1 hayesall hayesall  756 May 15 12:41 .viminfo

How do we learn more?

We could go on-and-on about ls, as there are literally thousands of variations just on the ls command: all configured through flags.

No one expects you to have every single variation on every single command memorized. If we were to allocate space for every variation: this chapter would meander on for thousands of pages.

But we do expect you to be able to look up the details when you need them. man pages, or manual pages are one of the best ways to look up this information without leaving the terminal.³

$ man ls

(Hint: Type Q to quit out of a man page.)

This chapter will cover what we feel are the absolute essentials: a self-contained, minimal set of commands that you need to be productive and accomplish your daily work. But there is no substitute for continuous learning and self-improvement.

Make Directories

mkdir directory » Make directory

On Unix systems, we will usually recommend you avoid using spaces. When typing commands, spaces are used to separate each part of command. So to create a directory:

$ mkdir daily-work

… will create a daily-work directory, but:

$ mkdir Daily Work

… creates two directories: Daily and Work.

$ ls
Daily  Work  daily-work

Remove Directories

rmdir directory » Remove directory

If we make a directory that we didn’t intend, or want to clean up, rmdir removes it:

$ rmdir Daily
$ rmdir Work

Make Files

touch file » Create a new file

Technically the touch command modifies timestamps, but its effect is to create new files if they do not exist:

$ touch notes.txt
$ ls
daily-work  notes.txt

One can also chain paths together using the / to create files inside of directories:

$ touch daily-work/step01.txt
$ touch daily-work/step02.txt
$ ls daily-work/
step01.txt  step02.txt

Remove Files

rm file » Remove, delete

⚠️ Caution! ⚠️

Most graphical user interfaces with friendly clickable buttons have Trash or a Recycling Bin. When the user deletes something, it goes in the bin, then the user must explicitly opt into emptying the trash.

Unix/Linux does not do this. When you tell your shell to remove something: it is gone, and there is no undo button.

When one can create files, one also needs to be able to remove them.

$ rm daily-work/step01.txt
$ ls daily-work/
step02.txt

But if one tries to remove something with something else inside of it: it’s an error:

$ rmdir daily-work/
rmdir: failed to remove 'daily-work/': Directory not empty
$ rm daily-work/
rm: cannot remove 'daily-work/': Is a directory

To remove a directory, everything inside of it, (and potentially everything inside everything that is inside of it—and so on): one must recursive remove with the -r flag:

$ rm -r daily-work
$ ls
notes.txt

Moving around inside the terminal:

cd path » Change Directory

cd is a program that takes a directory name as an argument, and changes into that directory. In a graphical file explorer, this is like double clicking a folder to navigate inside that folder.

If one creates a Classes directory with an i211 directory inside of it:

$ mkdir -p Classes/i211

Then the result is a file hierarchy. Some systems have a tree command (on Ubuntu/WSL: sudo apt install tree) that visualize these:

$ tree Classes/
Classes/
└── i211

Calling cd Classes/ will change your working directory. On Alexander’s: the absolute path becomes /home/hayesall/Classes, or relative path against the home directory is ~/Classes:

$ cd Classes
$ ls
i211

What happens if we type cd with no arguments?

$ pwd
/home/hayesall/Classes
$ cd
$ pwd
/home/hayesall

Therefore, you can always get home by typing cd with no arguments, or typing cd ~ (where the tilde ~ expands to your home directory).

Dot and Double Dot: The Relative Here and There

ls . » list files in here

cd .. » go there, to the parent directory

Every directory in a Unix-like system has two special resources inside of it: . and ...

The terminology for what these two things are called is inconsistent, so Alexander refers to them as here . and there .., because no matter where you are in some virtual space: there is always a here and a there. One may traverse from point A to point B by traversing the relative spaces between the two points.

• ~ - the home directory
• . - the current directory
• .. - the parent directory
• ../.. - the grandparent directory
• ../../.. - the great-grandparent directory

For example, if one creates five directories nested inside one another:

$ mkdir -p A/B/C/D/E
$ tree A
A
└── B
    └── C
        └── D
            └── E

… then one may reach C by with:

$ cd A/B/C
$ ls
D

… and then return home by following the chain of parent directories:

$ cd ../../..
$ ls
A

Since every directory has a . representing the current location on the file system, we can now think of ls as having the current directory . as a default argument. ls and ls . function the same:

$ ls .
A

From the rules of here and there, arbitrary complexity may be reached:

$ cd ./A/B/C/../C/../../B/././.
$ ls .
C

But even though these can be combined toward arbitrary complexity: one should strive to organize one’s file system in a logical way to avoid this. For example, in a web development project with hypertext markup language (HTML) and cascading style sheets (CSS), it’s common to organize files and directories like this:

$ mkdir -p some-html-project/css
$ touch some-html-project/{index,other}.html
$ touch some-html-project/css/main.css
$ tree some-html-project
some-html-project/
├── css
│   └── main.css
├── index.html
└── other.html

In this file hierarchy:

• when index.html references a CSS file, the relative path is ./css/main.css
• if index.html needs a link to other.html, the relative path is ./other.html since both are in the same directory

Which way to the root? Up ⬆️ or Down ⬇️ ?

Are you descending to the root, or ascending to the root?

Many will say ascending up to the root, and that cd .. is going up a directory. This is an artifact of how computer science textbooks historically drew trees: with the root node at the top and the leaf nodes at the bottom.

But this choice is arbitrary: it depends on how one views or visualizes hierarchy. Is it a foundation which everything rests upon? Or is it the peak for one to climb to, from which all authority flows?⁴

Edit a file in the command line

nano filename » Edit a file

nano is a simple text editor letting you write text inside of files without leaving the command line.

touch ~/Classes/i211/practice.txt
nano ~/Classes/i211/practice.txt

A basic interface will fill the space, typed text becomes part of the file:

• the cursor can only be moved using the keyboard, not the mouse
• when you are done you will run a set of commands to save, exit and return to the command line prompt

Control + X    (Exit)
Y              (Save modified buffer Yes No)
[Return]       (Return/Enter key will go back to the cursor)

Quickly view a file

cat filename » Concatenate the file to the terminal, showing the content

less filename » Preview the file

If you want to see if the file saved, or just check what is in a file, there are several ways to print out the contents in the command line.

If the file isn’t too long:

$ cat ~/Classes/i211/practice.txt
In i211, we are practicing with Unix and with
the nano text editor.

If you want a better interface, less is a terminal pager program where you can page up and down in the file, and use Q to quit back to what you were previously doing.

less ~/Classes/i211/practice.txt
q

Renaming, Moving, and Copying

mv old-name new-name » move or rename a resource from one name to another

cp some-name other-name » copy a resource from one place to another

Moving and renaming from the perspective of a file system are really the same fundamental operation: every file or directory has a name, and we’re changing that name from something to something else.

For example, if we accidentally type otes.txt when we intended to write notes.txt:

$ cd ~/Classes/i211
$ touch otes.txt
$ ls
otes.txt  practice.txt

Then we can rename the file by moving otes.txt to a new name: notes.txt:

$ mv otes.txt notes.txt
$ ls
notes.txt  practice.txt

Copying works the same way, but if the file contained content, then that content would be copied to a new location. Since practice.txt contained some text, copying the file somewhere should have the same text:

$ cp practice.txt assignment.txt
$ cat assignment.txt
In i211, we are practicing with Unix and with
the nano text editor.

If we want to copy a directory and everything inside of it—e.g. when copying files in a directory for a new class—we can modify how cp behaves with the recursive -r flag.

$ cd ~/Classes
$ ls
i211
$ cp -r i211 i210
$ tree ~/Classes
/home/hayesall/Classes
├── i210
│   ├── assignment.txt
│   ├── notes.txt
│   └── practice.txt
└── i211
    ├── assignment.txt
    ├── notes.txt
    └── practice.txt

2 directories, 6 files

Cleanup, Review, and Practice

We will not re-use any of the files or directories from this chapter, so everything from today is safe to remove:

$ rm -r ~/Classes

We’ll review and share some final notes before going into practice.

Terminal Shortcuts

Many text shells (and command line interface programs) come with these helpful features:

• Tab Completion - once the command or filename is typed in far enough that it is unique among possible names it could be, you can hit Tab ↹ on your keyboard and the filename will be completed for you
• History - commands are saved in your history. To cycle back to previous command: use the up arrow key ↑ on your keyboard as many times as needed

These two features are then made part of a host of keyboard shortcuts for text editing, autocompleting, and history searching.

🍎 Terminal Shortcuts on macOS

macOS shortcuts vary slightly depending on the shell that came with your computer. We list variants in the table, or you can use the dropdown menus at the top to check for your computer.

Pathnames

The file structure we learned here today is repeated throughout the coding world, and understanding how pathnames work is important to creating projects later in the course.

A pathname is, you guessed it, what pwd prints out in our terminal. It’s a text representation of the data structure used for organizing files and directories. And once you see it, you’ll see it everywhere there is code.

Even as a URL in your browser!

Review

Commands from this chapter have two main uses: those that give us information about the state of our system,

and those that mutate (create, remove, update) the system to fit our needs:

Practice Exercises

01 Simple Hierarchy

Recreate this file hierarchy using mkdir, touch, and maybe cd

unix-practice
└── f01
   ├── f1.txt
   └── f2.txt

mkdir unix-practice
mkdir unix-practice/f01
touch unix-practice/f01/f1.txt
touch unix-practice/f01/f2.txt

mkdir -p unix-practice/f01
touch unix-practice/f01/{f1,f2}.txt

02 Website Hierarchy

Static websites may be built from hypertext markup language (HTML), cascading style sheets (CSS), and JavaScript (JS). Sites typically have extra configuration files (e.g., robots.txt, Sitemaps) to help online services like search engines index the site. Reproduce the following website template using mkdir, touch, and possibly cd.

unix-practice
└── some-empty-site
   ├── css
   │  ├── main.css
   │  └── reset.css
   ├── index.html
   ├── js
   │  └── main.js
   ├── robots.txt
   └── sitemap.xml

mkdir unix-practice/some-empty-site
mkdir unix-practice/some-empty-site/css
mkdir unix-practice/some-empty-site/js
cd unix-practice/some-empty-site/css
touch main.css reset.css
cd ..
touch index.html robots.txt sitemap.xml
touch js/main.js
cd ../..

mkdir -p unix-practice/some-empty-site/{css,js}
touch unix-practice/some-empty-site/css/{main,reset}.css
touch unix-practice/some-empty-site/js/main.js
touch unix-practice/some-empty-site/{index.html,robots.txt,sitemap.xml}

03 Python Hierarchy

A Python package is a collection of Python code and configuration files informing a package installer (e.g. pip) where code is and how that code is loaded in order to be used in a downstream dependency. Reproduce the file hierarchy of a typical Python project inside the unix_practice folder, using mkdir, touch, and cd as needed:

unix-practice
└── empty-python-project
   ├── my_project
   │  ├── __init__.py
   │  ├── __main__.py
   │  └── tests
   │     ├── __init__.py
   │     └── test_setup.py
   ├── pyproject.toml
   ├── README.md
   └── requirements.txt

mkdir unix-practice/empty-python-project
mkdir unix-practice/empty-python-project/my_project
mkdir unix-practice/empty-python-project/my_project/tests
cd unix-practice/empty-python-project
touch pyproject.toml
touch README.md
touch requirements.txt
cd my_project
touch __init__.py
touch __main__.py
cd tests
touch __init__.py
touch test_setup.py
cd ../../../..

mkdir -p unix-practice/empty-python-project/my_project/tests
touch unix-practice/empty-python-project/{pyproject.toml,README.md,requirements.txt}
touch unix-practice/empty-python-project/{my_project,my_project/tests}/__init__.py
touch unix-practice/empty-python-project/my_project/__main__.py
touch unix-practice/empty-python-project/my_project/tests/test_setup.py

04 Julia Hierarchy

Julia is a relatively new programming language developed with scientific programming in mind. Reproduce the file hierarchy used in a typical Julia project:

unix-practice
└── empty-julia-project
   ├── docs
   │  ├── make.jl
   │  └── Project.toml
   ├── Project.toml
   ├── README.md
   ├── src
   │  ├── main.jl
   │  └── types.jl
   └── test
      ├── Project.toml
      ├── runtests.jl
      └── test_types.jl

mkdir unix-practice/empty-julia-project
cd unix-practice/empty-julia-project
mkdir docs src test
touch Project.toml README.md
cd docs
touch make.jl Project.toml
cd ../src
touch main.jl types.jl
cd ../test
touch Project.toml runtests.jl test_types.jl
cd ../../..

mkdir -p unix-practice/empty-julia-project/{docs,src,test}
touch unix-practice/empty-julia-project/README.md
touch unix-practice/empty-julia-project/{.,docs,test}/Project.toml
touch unix-practice/empty-julia-project/docs/make.jl
touch unix-practice/empty-julia-project/src/{main,types}.jl
touch unix-practice/empty-julia-project/test/{runtests,test_types}.jl

05 Java Hierarchy

Java is known for using deeply-nested paths. (Hint: mkdir -p creates all intermediate paths, allowing you to string together several/different/directories at once). Reproduce a Java file hierarchy:

unix-practice
└── empty-java-project
   ├── build.gradle
   ├── README.md
   └── src
      ├── main
      │  └── java
      │     └── com
      │        └── hayesall
      │           └── Main.java
      └── test
         └── java
            └── com
               └── hayesall
                  └── Test.java

mkdir -p unix-practice/empty-java-project/src/main/java/com/hayesall
mkdir -p unix-practice/empty-java-project/src/test/java/com/hayesall
cd unix-practice/empty-java-project
touch build.gradle README.md
touch src/main/java/com/hayesall/Main.java
touch src/test/java/com/hayesall/Test.java
cd ../..

mkdir -p unix-practice/empty-java-project/src/{main,test}/java/com/hayesall
touch unix-practice/empty-java-project/{build.gradle,README.md}
touch unix-practice/empty-java-project/src/main/java/com/hayesall/Main.java
touch unix-practice/empty-java-project/src/test/java/com/hayesall/Test.java

06 Text Trip Planning

Now that we’re comfortable (1) creating files and directories, and (2) navigating with cd; let’s add file editing with nano into our workflow.

We’re going on a trip. Let’s create files and directories to house our planning notes. Start with:

mkdir ~/TripPlanning

then:

1. navigate into the TripPlanning directory
2. create a file called snacks.txt
3. edit snacks.txt, adding your favorite snacks
4. save and exit nano

Now add two directories: Destinations and Ideas.

TripPlanning
├── Destinations
├── Ideas
└── snacks.txt

Create files in Destinations and Ideas for some places you want to go and some ideas for what you might do while you’re there. Here are some ideas if you’re stuck:

TripPlanning
├── Destinations
│  ├── Gershwin-Theatre.txt
│  ├── Met-Museum-of-Art.txt
│  └── Statue-of-Liberty.txt
├── Ideas
│  ├── drag-brunch.txt
│  └── roller-skating.txt
└── snacks.txt

Self-reflection:

• Can you successfuly add some text to each .txt file using nano?
• Can you move up a level from Ideas back to TripPlanning?
• Can you change from Ideas to Destinations in one line?
• How about getting back home?

When you have completed the practice, you can delete the practice directory:

rm -rf ~/TripPlanning

Exit the terminal when you’re finished, either by:

• typing: exit + ↵ Enter
• or send an end-of-transmission ␄ with: ^ Ctrl + D

Stuck?

Chat with us on Discord, in office hours, or send us an email!

Footnotes

Further Reading

• “The Art of Unix Programming,” Eric S. Raymond. CC BY-ND 1.0
• List of Unix Commands