II - Files and Directories

Files and Directories

Files store some sort of information: data, source code for scripts or programs, images, video, and so forth.

There are two basic types of files:

• Text (documents, code)
• Binary (images, executables)

The Unix shells ignore file extensions, but software might require certain extensions. This includes compilers (.cxx, .f90 and so forth), interpreters (.py, .R, etc.), image and video tools (.png, .jpg, .mp4). Since the format of the file extension depends on the expectations of software or on user preference, there is no rule limiting the number of characters, but most consist of one to three characters.

Directories are collections of files and other directories. Often they are called folders, but directory is generally the preferred (and historical) term in Unix.

Directories that are stored within another directory are subdirectories of the parent.

Both files and directories have “metadata” associated with them such as name, timestamps, and permissions.

The names of Unix files and directories should contain no spaces.

$ls 

'10. directories.md'
'15. streams.md'

The quotes indicate that a space is part of the file name. While most modern Unix tools can handle spaces, the shell does not always do so, and special precautions must be taken to avoid surprises. For that reason, underscores or hyphens are preferred instead of spaces.

Paths

In most operating systems, all files and directories are located with a path. The “path” is the “full name” of every file & directory.

In a Unix-based operating system, the files are organized into a tree structure (the method to store and organize the files is called filesystem, e.g. ext3,ext4, NTFS, etc). The tree is “upside down” because the root directory is at the top, and directories branch off from there. The root directory is indicated with a forward slash /.

The home directory is the usual name on Linux and similar Unix-based operating systems for the folder that holds user directories and files. On macOS it is called User. On both, the separator between branches of the tree is the forward slash.

/home/mst3k/myfile
/Users/Misty Tea/Documents/Homework.pages

Windows files and folders also have paths. In Windows, drive letters or volumes are the top-level folders, and usually there is more than one. User files are in Users, similar to macOS.

Absolute and Relative Paths

Paths may be absolute or relative.

An absolute path is path to a file or folder starting at the root. On Unix it will begin with /, to designate the root.

An absolute path is guaranteed to get you to the location you want.

A relative path is the path to a file starting at the current location. We use the notation . (a single period) for the current working directory, and .. (two periods) for the parent of the current working directory.

Examples

Absolute paths:

/home/mst3k/file.txt 
/home/mst3k/files/file.txt
/home/mst3k/projects/project1
/home/mst3k/projects/project1/output.txt

Note that /home/mst3k/file.txt and /home/mst3k/files/file.txt are different files unless you explicitly link them.

Relative paths. Suppose we are in the /home/mst3k/files folder.

./file.txt		
../files/file.txt

The relative path to file.txt in files from the project1 folder is

../../files/file.txt

Tilde Notation

In Unix the tilde ~ stands for the user’s home directory, e.g. /home/mst3k.

ls ~
ls ~/files

File Commands

ls

ls lists files in a directory.

$ls 

With no argument, listing the entire contents of the current working directory is assumed.

Like most Unix commands, it has many options. They may be combined.

$ls -ltr ./projects

cp

cp to copy a file.

$cp file1.txt file2.txt
$cp mymod.py ../projects/python_code

Commonly-used options:

$cp -r /share/resources/tutorials/rivanna-cl ~

mv

mv to rename or move a file.

$mv file1.txt file2.txt
$mv mymod.py ../projects/python_code

Options for mv are similar to cp.

rm

rm to remove a file or directory

$rm file1.txt 
$rm file1.txt data.csv

Once the file is removed, it is gone and can no longer be accessed.

Options for rm are similar to cp.

Directory Commands

cd

cd change directory

$cd 

With no argument, cd moves to the user’s home directory. If you are hopelessly lost, you can always type cd and start over.

$cd ../projects/project1

mkdir

mkdir to create a directory. Works relative to current working directory.

$mkdir new_project

or

$mkdir new_project
$mkdir ~/projects/new_project/src

rmdir

rmdir to remove an empty directory. Does not affect directories with content.

$rmdir ~/projects/new_project/src

rm -rf

rm -rf to force the removal of a directory and all its contents, including subdirectories.

$rm -rf ~/projects/project1/src

cp -r

Directories are copied with the cp -r command previously discussed.

Exercise

Start a terminal on a loginnode by whatever means you prefer. Type (the $ is the prompt and is not typed):

$echo $SHELL
$pwd
$mkdir test_dir
$cd test_dir
$ls ..
$mkdir sub_dir
$ls

Q1: What is the full path of your home directory?

$cd ~
$pwd

Q2: Delete the sub_dir folder.

$rmdir ~/test_dir/sub_dir

Q3: Copy the directory /share/resources/tutorials/rivanna-cl/shakespeare into your home directory.

$cp -r /share/resources/tutorials/rivanna-cl/shakespeare ~