Useful Linux commands

From Earlham CS Department
Revision as of 17:49, 15 May 2025 by Tdjones22 (talk | contribs) (Added fsck commnd)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

The one-stop shop for all your Linux command needs. Most commands listed will work on our machines, though that can vary based on installed packages and/or operating systems.

For almost every command, you can use the --help argument, <command> --help to get detailed information about the command as well as a list of its arguments.

File and Directory Management

ls

Lists all files and directories

  • ls -a - List all files, including hidden ones like .ssh, .bashrc, and so on.
  • ls -l - Include extra file information like permissions, owner, groups, modified date, and more.
  • ls -s - List size of files in blocks.
  • ls -S - List by size, largest first.

cd

Used to change the directory.

  • cd - When no path is specified, you will move to the user's home directory. As root, you will move to /root/. As your user, you will move to /eccs/home/<username>/.
  • cd /path/to/file - When changing directories, if you're not in the immediate parent directory, you need to specify the full path. For example, if my current working directory is /etc, I can cd jupyterhub since it's within /etc. However, if my current working directory is /bin, I would have to specify the full path for /etc/jupyterhub.
  • cd . and cd .. - The . and .. are used for adjacent movement. cd . "moves" you into the current directory, so basically no movement. cd .. moves you backwards one directory.
  • cd - - The dash specification takes you to the previous working directory regardless of adjacency.

pwd

Displays the current working directory. This can be usual on certain machines that don't display the full path of your current working directory, like Whedon.

root@w0 plugins # pwd
/etc/nagios/plugins

In the above example, my current working directory is /etc/nagios/plugins, but the prompt only lists it at nagios.

mkdir

Used to create directories.

  • You can use mkdir directory_name to create it in your current working directory.
  • Additionally, you may specify the full path to create the directory somewhere else mkdir /path/to/directory_name.

rmdir

Used to remove empty directories.

  • You can ONLY remove empty directories. You cannot specify recursive or force.

mv

Used to move and rename files.

  • This command uses a mv /path/to/file /path/to/destination syntax. It follows the same rules as cd in that you need to specify the full path unless moving to the current working directory or an adjacent directory.
  • To rename a file, you can "move" it into a new one. mv old.txt new.txt will put the contents of old.txt into the newly created new.txt. This action will delete old.txt. NOTE: This WILL overwrite the file if you move it into a preexisting one.
  • mv -i - If you're worried about overwriting files, you can use the -i specification. This will prompt you for confirmation before overwriting.

rm

Used to remove files and/or directories. Unlike rmdir, you can use rm to remove directories. Even ones that aren't empty.

  • rm -d - Essentially the same as rmdir; it can only remove empty directories.
  • rm -r - Recursively removes files and directories from a directory, including itself.
  • rm -rf - The same process as rm -r, except it bypasses any locks or protections on files and deletes them. Only use this command if you are certain you understand what you're deleting, especially if you're root.
  • The * isis a powerful wildcard in Linux that allows you to match multiple files efficiently when using commands like rm. For example, take a look at this directory:
tdjones22@bowie:~/directory$ ls
123.txt  131.txt  14.txt  1file.txt  208.txt  263.txt  62.txt code.py code.rb config.conf

Let's say you want to remove every file that starts with the number 1. You can use rm *1 to remove any file matched with the first character being 1.
Files left after the command:

208.txt  263.txt  62.txt code.py code.rb config.conf

Let's say you want to remove files with a specific extension. You can use rm *.txt to remove every file with the .txt extension.
Files left after the command:

code.py code.rb config.conf

cp

Used to copy files and/or directories. Similar to rm in that you need to specify recursive when copying a non-empty directory. It follows a cp /path/to/file /path/to/destination syntax.

  • cp -r - Recursively copies a directory; the only way, using cp, to copy a directory that isn't empty.

find

A tool to find files using many criteria including name, type, contents of file, and more. A common syntax for this command is find /starting/directory -specification "name_of_file.txt".

  • find / -name "name.txt" - A typical usecase for find is searching by file name. If I've lost track of a file named found_file.txt, I can search the entire machine for it using this command:
tdjones22@bowie:~$ find / -name "found_file.txt" 2>/dev/null
/earlhamcs/eccs/users/tdjones22/directory/found_file.txt

This command will usually take some time to execute fully, so don't be alarmed when it hangs for a few seconds.


The starting directory was specified as / which is just the parent directory, meaning it searches the entire file system. The search type was specified as -name, which looks for matching named files. You can additionally add 2>/dev/null to silence all of the permission errors you'll get when searching all of the machine's directories.

  • find / -name "*.txt" -exec grep -iIl "contents to search for" {} + 2>/dev/null - If you know a portion of the contents of a file, but you don't know the name, you can use find in tandem with grep to find it. Though, it's far easier and more convenient to only use grep.

Let's say I have a missing text file, but I know it contains "solar power", I can find it like so:

tdjones22@bowie:~$ find / -name "*.txt" -exec grep -RiIl "solar power" {} + 2>/dev/null
/eccs/home/tdjones22/test_find.txt

Like the previous command, this will take a hot minute to complete.


The starting directory was specified as / which is just the parent directory, meaning it searches the entire file system. We're looking for a file by its name specified as "*.txt", which looks for any files with the .txt extension. The -exec grep -iIl "solar power" {} + section is what searches through the file's contents. The grep arguments specify the following: -l - ignore case, -I - ignore binary files, and -l - display file name. You can additionally add 2>/dev/null to silence all of the permission errors you'll get when searching all of the machine's directories.

File Permissions and Ownership

chmod

Used to add, remove, or change file permissions. You normally specify permissions using an octal format, but you may also use symbolic as well. Here are some common chmod commands; full access for a user implies read, write, and execute ability:

  • chmod 600 | rw------- - The owner can read/write, but there is no access for others. Some use cases are private files and SSH keys.
  • chmod 644 | rw--w-r-- - The owner can read/write, but others can only read. Useful for config files and public web files.
  • chmod 660 | rw-rw---- - The owner and group can read/write, but there is no access for others. Useful for files used specifically in group collaboration.
  • chmod 664 | rw-rw-r-- - The owner and group can read/write, but others can only read. Useful for files used specifically in group collaboration.
  • chmod 700 | rwx------ - The owner has full access, but there is no access for others. Useful for private scripts (since you need the ability to execute a script) and sensitive directories.
  • chmod 755 - | rwxr-xr-x The owner has full access, but others can only read and execute. Useful for certain executables and web directories.
  • chmod 775 | rwxrwxr-x - The owner and group have full access, but others can only read and execute. Useful for group-managed directories.
  • chmod 777 | rwxrwxrwx - Everyone, and I mean everyone, has full access. So...don't use this unless absolutely necessary.

chown

Used to change which user, group, or both own files and directories.

  • chown user file - Change the owner of file to user.
  • chown user:group file - Change both owner and group of file.
  • chown :group file - Change only group of file.
  • chown -R user:group directory/ - Recursively change owner and group for all files in directory/.

chgrp

Used to change only the owner of both files and directories.

  • chgrp group file - Change the group of file to group.
  • chgrp group directory/ - Change the group of directory/.
  • chgrp group file1 file2 file3 - Change the group of multiple files.
  • chgrp -R group directory/ - Recursively change group for all files, including the directory itself, in directory/.

stat

Used to view various stats about files and directories such as size, blocks, type, inode, UID/GID, last access, creation date, and more. Let's see what happens when we run stat on the file test_find.txt:

tdjones22@bowie:~$ stat test_find.txt
  File: test_find.txt
  Size: 27              Blocks: 8          IO Block: 4096   regular file
Device: fe04h/65028d    Inode: 25452759    Links: 1
Access: (0644/-rw-r--r--)  Uid: (11035/tdjones22)   Gid: (10000/sharedusers)
Access: 2025-03-28 13:42:16.638585831 -0400
Modify: 2025-03-28 11:31:47.781444104 -0400
Change: 2025-03-28 11:31:47.781444104 -0400
Birth: 2025-03-28 11:31:47.781444104 -0400

Let's talk about the output:

  • Size: 27 - The file is 27 bytes.
  • Blocks: 8 - The file occupies 8 blocks on the disk.
  • regular file - This is a "regular file", not a directory, symlink, etc.
  • Device: fe04h/65028d - The device ID of where the file is stored.
  • Inode: 25452759 - The inode number assigned to the file. Every file and directory has a unique inode number.
  • Links: 1 - The number of hard links to this file.
  • (0644/-rw-r--r--) - Permissions in octal and symbolic format.
  • Uid: (11035/tdjones22) Gid: (10000/sharedusers) - The user ID (Uid) of the owner and the group ID (Gid).
  • Access: 2025-03-28 13:42:16.638585831 -0400 - The last time the file was read.
  • Modify: 2025-03-28 11:31:47.781444104 -0400 - The last time the file's contents were modified.
  • Change: 2025-03-28 11:31:47.781444104 -0400 - The last time metadata (permissions, ownership, etc) was changed.
  • Birth: 2025-03-28 11:31:47.781444104 -0400 - When the file was first created.

Running this command on a directory displays the same format of output.

Process Management

ps

Used to display information about running processes. There are two common argument options when using ps: Unix options, which need to be preceded by a dash, and BSD options, which can be grouped and do not need a dash.

ps is a rather deep command with many specifications and shorthand syntax that is not necessarily intuitive. Here is the link to the official Bash documentation for PS1, which goes heavily into detail about the command with useful quick links at the top of the page.

Here are some common uses:

  • ps - Shows only the processes running in your current terminal session.

This is typical output for running ps alone:

    PID TTY          TIME CMD
1980517 pts/0    00:00:00 bash
1980970 pts/0    00:00:00 ps
  • PID - The Process ID (PID) is the unique number that identifies a process.
  • TTY - TTY indicates the computer terminal that executed the process.
  • TIME - The total CPU time the process has used since it started.
  • CMD - The full command used to initiate the process. Includes paths and arguments.
  • ps -e/ps -A - Lists all of the running processes on the machine using the standard syntax.
  • ps -ef - Lists all of the running processes using the full format. Let's look at what it adds on top of the standard ps:
  • UID - The User ID (UID), usually a username, that owns the process.
  • PPID - The Parent Process ID (PPID).
  • C - The CPU usage in percentage over the process's runtime.
  • STIME - The time in HOUR:MINUTE format that the process started.
  • ps -u user - List all of the running processes for a specified user.
  • ps -eH - List all of the running processes in a hierarchical view. It visually indents processes in a human-readable syntax.
    920 ?        00:00:29   sshd
1980491 ?        00:00:00     sshd
1980516 ?        00:00:00       sshd
1980517 pts/0    00:00:00         bash
1983776 pts/0    00:00:00           ps

This is the process hierarchy of my bash shell (1980517), via ssh, running the ps -eH command.

  • ps aux - List all processes (a) for all users (u) using the wide syntax (x).
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
tdjones+ 1984316  0.0  0.0   9984  3488 pts/0    R+   14:17   0:00 ps aux

Let's talk about the wide syntax columns:

  • %CPU - CPU usage.
  • %MEM - Memory usage.
  • VSZ - Virtual memory size in KB. It includes all code, data, and shared libraries.
  • RSS - Resident Set Size (RSS) is the actual physical memory used in KB.
  • STAT - The state of the command.
  • ps -o - The -o argument allows you to specify the format of the output. Here is a command with all of the possible specifications:
ps -o user,pid,ppid,%cpu,%mem,vsz,rss,tty,stat,start,time,cmd

You can specify these in any order.

top

Shows an updating list of currently running processes, along with their CPU and memory usage. When running top in your terminal, you'll be prompted with a command line interface (CLI) like so:

Tasks: 276 total,   3 running, 273 sleeping,   0 stopped,   0 zombie
%Cpu(s): 25.2 us,  2.1 sy,  0.0 ni, 72.0 id,  0.0 wa,  0.0 hi,  0.7 si,  0.0 st
MiB Mem : 193403.2 total,  81554.2 free,   2989.2 used, 108859.8 buff/cache
MiB Swap:  66756.0 total,  66213.6 free,    542.4 used. 189014.0 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
1610433 qkalaya+  20   0   13836   5796   3088 R  94.1   0.0  15822:10 python3.10
1610629 qkalaya+  20   0   13832   5808   3088 R  94.1   0.0  15821:10 python3.10
2508275 mgcithi+  20   0   11.2g  78884  44700 S   5.9   0.0   1:19.80 node
2553235 tdjones+  20   0   10520   3952   3364 R   5.9   0.0   0:00.02 top

Here are some handy controls once you're in the CLI:

  • q - Quits and exits the CLI. Ctrl + C also works for exiting, just not as gracefully, perhaps.
  • k - Kills the process identified by an input process ID (PID). A more interactive and readable alternative to using kill
  • r - 'Renice' a process; change the processes's priority. The 'niceness value' of a process determines how often it receives CPU time relative to other processes. The actual value itself ranges from -20 to 19, with -20 being the highest priority and 19 being the lowest.
  • P - Sort processes by CPU usage. This is the default when opening top.
  • M - Sort processes by memory usage.
  • T - Sort processes by time running.

Additionally, if you want or need to take a snapshot of top's data at a single moment, you can run top -b. You can also save its output to a file using top -b -n 1 > output.txt. It's important to specify -n 1, which tells it to capture the batch once; otherwise, it'll loop forever.

kill

Used to kill a single running process using its process ID (PID). kill has a few options that you'll commonly use. Here are some:

  • kill -15 PID - This is the same as running kill without any specifications. It gracefully shuts down the process, allowing it to save its current state, close corresponding files, remove temporary files, log its shutdown process, and notify other processes/systems.
  • kill -9 PID - This really kills a process. It skips all cleanup procedures and forcefully stops the process and anything associated with it.

pkill

Used to kill any processes that match the input name. Unlike kill, this can terminate multiple processes. Here are some common uses:

  • pkill process_name - This will cleanly terminate the specified process.
  • pkill -f process_name - This allows you to terminate a process based on any matching string within the full command line. For example, if I run pgrep -fa python3, I can see all of the processes that would be terminated if I swapped pgrep for pkill:
tdjones22@bowie:~$ pgrep -fa python3
899 /usr/bin/python3 /usr/bin/fail2ban-server -xf start
1569 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
1610433 /usr/local/bin/python3.10 /eccs/home/qkalaya24/sll.py
1610629 /usr/local/bin/python3.10 /eccs/home/qkalaya24/sll.py

I'm able to kill a process based on it using any version of python3. This is true for anything you see listed in the full process.

  • pkill -u user process_name - This will only match processes owned by the specified user.
  • pkill -x process_name - This kill a process only if it's name matches exactly.
  • pkill -n process_name - If there are multiple matching processes, it will kill the newest matching one.
  • pkill -o process_name - If there are multiple matching processes, it will kill the oldest matching one.
  • pkill -9 process_name - Force quits the process.

nice

Used to adjust how much CPU time a process gets relative to other processes. Every process has a nice value between -20 (highest priority) and 19 (lowest priority). Niceness levels below 0 must be set by root.

  • nice -n value command - When running nice, it means you're starting the command. For example, nice -n 15 python3 code.py sets the niceness value and then executes the following command.

renice

Used to adjust how much CPU time an already running process gets relative to other processes. Every process has a nice value between -20 (highest priority) and 19 (lowest priority). Niceness levels below 0 must be set by root. Renice only accepts process IDs (PIDs).

  • renice 10 -p 1234 - This will change the niceness level of the process correlated with PID 1234 to a level of 10.

User Management

whoami

This command prints the username associated with the effective ID. If I run whoami as my stand user, this is the output I'd expect:

tdjones22@bowie:~$ whoami
tdjones22

Simple and straight to the point.

However, for a fun example of how effective UID (EUID) works vs real UID (RUID), run whoami as your user with sudo:

tdjones22@bowie:~$ sudo whoami
root

who

Lists current users who are logged in with information regarding their username, the date and time of login, and the IP address they were connected to when logged in.

w

Lists current logged-in users with additional information such as username, from address, login date, idle time, CPU usage, and the last command they used.

id

This command shows your user ID, group ID, and the IDs of any other groups or privileges you have. Running the command with no specification defaults to showing your IDs:

tdjones22@hopper:~$ id
uid=11035(tdjones22) gid=10000(sharedusers) groups=10000(sharedusers),1210(cybersecurity),5050(sudoers)

However, you may specify a username and see their IDs as well:

tdjones22@hopper:~$ id charliep
uid=10042(charliep) gid=10000(sharedusers) groups=0(wheel),121(software),1007(bccd),116(mothur),122(fieldscience),1208(production),120(staff),170(hip),173(pedagogical),176(cs-department),1030(cvs),5050(sudoers),1108(fieldsci),1109(webdev),1107(dpr),1115(nbgrader),1200(csfaculty),1206(cluster),10000(sharedusers)

passwd

This command allows you to change the password of the specified user. When changing your own password, you will be prompted to enter your old one before you can set the new one:

tdjones22@hopper:~$ passwd tdjones22
Enter login(LDAP) password:
New password:

If you or someone you know has lost their password, you can bypass the LDAP authentication by either being logged in as root or using sudo:

tdjones22@hopper:~$ sudo passwd tdjones22
New password:

su

This command allows you to switch to another user, assuming you have privileges and/or the password. This is a vital command in the System Administration world because it's the most common way to become root:

tdjones22@hopper:~$ sudo su -
root@hopper:~#

You'll notice that we're using - in place of root. Here's what happens if you switch user directly to root:

tdjones22@hopper:~$ sudo su root
root@hopper:/cluster/home/tdjones22#

It preserves the current working directory you were in before switching to root, which, in my case, was my personal home directory. cd - takes you to the directory you last visited, but in the context of sudo su -, it takes you to root's home directory /root.

sudo

Short for superuser do, sudo is one of the most powerful tools in the System Administrator toolbox. When logged into an account with sudo/sudoers access, you can run any command with root privileges. It temporarily changes your effective UID (EUID) to match that of root's EUID, giving you root privileges.

Disk and Filesystem Management

df

Standing for disk free, df provides information on disk space utilization, specifically for filesystems.

  • df -h - Perhaps the most common way to use df, this specification converts the bytes into *human* readable syntax: MB, GB, TB, and so on.
tdjones22@bowie:~$ df -h
Filesystem                     Size  Used Avail Use% Mounted on
udev                            95G     0   95G   0% /dev
tmpfs                           19G  1.6M   19G   1% /run
/dev/mapper/virtual0-root      274G  186G   75G  72% /
tmpfs                           95G  2.1M   95G   1% /dev/shm
tmpfs                          5.0M     0  5.0M   0% /run/lock
/dev/mapper/virtual0-docker   1015G  300G  665G  32% /docker
/dev/mapper/virtual0-eccs--lv  4.0T  2.7T  1.2T  70% /earlhamcs
/dev/mapper/virtual0-postgres 1007G  457G  499G  48% /postgres
tmpfs                           19G     0   19G   0% /run/user/11162
tmpfs                           19G     0   19G   0% /run/user/11170
tmpfs                           19G     0   19G   0% /run/user/11035

Let's talk about the output:

  • Filesystem - The name of the mounted filesystem.
  • Size - The total size of the filesystem.
  • Used - The amount of space occupied in the filesystem.
  • Avail - The remaining usable space in the filesystem.
  • Use% - The percent of the filesystem used.
  • Mounted on - The directory the filesystem is mounted on.
  • df -h file/directory - You can run df -h on a file or directory to get information about the filesystem it lives in.
tdjones22@bowie:~$ df -h output.txt && echo "-------------" && pwd
Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/virtual0-eccs--lv  4.0T  2.7T  1.2T  70% /earlhamcs
-------------
/eccs/home/tdjones22

du

Standing for disk usage, du provides information on the amount of disk usage for files and directories.

  • du -h - Perhaps the most common way to use du, this specification converts the bytes into *human* readable syntax: MB, GB, TB, and so on.
  • du --max-depth=<number> - When running du, it displays disk usage for every directory recursively, meaning if your current working directory is somewhere higher up in the hierarchy, you'll get a huge slurry of output. --max-depth=<number> defines how many recursive steps the command will take:
tdjones22@bowie:~/CS325$ du -h --max-depth=1
8.0K    ./.vscode
30M     ./Lab2
5.7G    ./head_node_backup
5.8G    .

This shows the disk usage for the base-level directories in your current working directory. NOTE: du -h -d 0 is the same as du -h --max-depth=0. I prefer the additional context of --max-depth=0.

You can also use the depth specifier to see the disk usage for only your current working directory:

tdjones22@bowie:~/CS325$ du -h --max-depth=0
5.8G    .
  • du -a - By default, du does not display disk usage of files unless the file is specified du file. However, the -a argument displays disk usage for all, including directories and files.
tdjones22@bowie:~/CS325$ du -h -a --max-depth=1
4.0K    ./web_check_v2.txt
4.0K    ./artificial_load.md
48K     ./artificial_load.pdf
4.0K    ./jumbo_frames.md
4.0K    ./loop.sh
8.0K    ./.vscode
4.0K    ./website_check.sh
4.0K    ./config_manifest.md
30M     ./Lab2
5.7G    ./head_node_backup
28K     ./jumbo_frames.pdf
8.0K    ./final_study_guide.md
40K     ./config_manifest.pdf
4.0K    ./test.sh
4.0K    ./website_check_v2.sh
5.8G    .

mount

Used to mount filesystems. This works on filesystems being sent through Network File System (NFS) as well as filesystems on mounted disks (HDDs, SSDs, USB Flash Drives, Memory Cards, etc).

  • mount -a - The -a specification tells your machine to mount all available filesystems listed in /etc/fstab. This is useful for system administration because it's common to mount multiple filesystems being exported from multiple machines.
  • mount /path/to/filesystem - Specifying the filesystem you want to mount means you will only mount that filesystem. This is the command you'll want to use when mounting a filesystem from a device.

In CS325 (Systems Engineering and Administration), we set up a cluster using Raspberry Pi 5's. The Pi's have two main methods of storage, a micro SD card and a mini NVMe slot. The SD card we were using was faulty, resulting in it crashing and us having to wipe the head node three times. However, when it bricked the final time, I was able to take the SD card, pop it into a machine in Lovelace, and mount it using mount /dev/sdc2. The boot partition sdc1 had become corrupted, but the root partition with all of our data was salvageable. All of this to say, mount is an extremely useful tool in the Linux world.

umount

Used to unmount filesystems. This works on filesystems being sent through Network File System (NFS) as well as filesystems on mounted disks (HDDs, SSDs, USB Flash Drives, Memory Cards, etc). It's like the "Eject" option you select on Mac or Windows when removing something like an external hard drive to ensure nothing bad happens.

  • umount -a - The -a specification tells your machine to unmount all filesystems listed in /etc/fstab.
  • umount /path/to/filesystem - Specifying the filesystem you want to unmount means you will only unmount that filesystem. This is the command you'll want to use when unmounting a filesystem from a device.

fsck

Standing for File System Consistency Check, this is used to check and repair filesystems. NOTE: Do NOT run this command on a mounted filesystem as it can very easily corrupt the filesystem data.

  • It starts by reading the superblock of the filesystem, which is a metadata structure that contains vital information such as the size, type, status, and layout.
  • After reading through the superblock, it confirms filesystem consistency by checking:
  • Inode table - Each file/directory has an inode with metadata.
  • Block allocation - Ensuring no two files claim the same data block.
  • Directory structures - Ensuring that directory entries point to valid inodes.
  • Link counts - Comparing actual hard links versus what the inode metadata says.
  • Free block/inode bitmaps - Ensuring they reflect what's actually in use.
  • If the command detects no issues, it will report some information about the filesystem and conclude its investigation. If it detects errors, it will prompt you for permission to fix them.

I highly recommend doing more intensive research on fsck before using it, as it can be destructive if used improperly.


-- Major Overhaul: March 2025 by Tyler Jones [tdjones22] --