Copy, Move, Delete, and Examine Unix Files ========================================== .. image:: imgs/filesys1.jpg :align: center Video for this section ====================== Video for this section is not available yet. Copying Files and Directories ============================= The ``cp`` command ------------------ ``cp file1 file2`` is the command which makes a copy of ``file1`` and called ``file2`` . In this case, both files would be in the current working directory. That's pretty simply. You might do ``cp mycode.py mycode.backup23sept`` to make a backup copy of your paper before you editted it Other forms are: ``cp file1 dir`` This form places a copy of ``file1`` in *directory* ``dir``. Notice that it has the same form as copying a file and giving it a new name, but the second **argument** to ``cp`` is a *directory* destination. This usage copies the *file* ``file1`` to the *location* ``dir``, but uses the same name as the original. Also, you can do: ``cp file1 file2 file3 dir`` which places copies of the three files ``file1``, ``file2``, and ``file3`` in directory ``dir``. Notice that if the last argument to cp is an existing directory , all preceding files in the argument list will be placed there. Both files and directories can have full pathnames if we want to copy files to different places in the file system. If I were copying the three files above (named ``file1``, ``file2``, and ``file3``), I would use a **Wildcard** match with ``*``, and do it like this ``cp file* dir`` This form matchs all files whose names begin with ``file``. | Time for an example. In your ``unixplay/`` dir, in one of the subdirectories created when you unpacked the tar file, you should find a file named ``testfile``. Verify this by listing the directory ``unixplay/``. Now ``cd`` to your home directory, and type: ``fin testfile`` I want to show you how your *find alias* works. Try it again by this time do: ``fin test\*`` (You NEED the ``\`` in front of the ``*``). That's how you give a *wildcard* match to the ``fin`` alias. Ok, let's play. ``cd`` into ``unixplay/``. ------------------ Make a copy of **testfile** called **testfile2**, by typing ``cp testfile testfile2`` Verify by listing that you now have 2 files: **testfile** and **testfile2**. Now let's take a file stored in ``unixplay/`` and use the ``cp`` command to put a copy in your ``newdir/`` directory. First, ``cd`` into your ``newdir/`` directory. It's below your home directory. Wherever you are, you can get there by typing the full pathname (using the ``~`` alias for your home dir): ``cd ~/newdir`` Then at the prompt, type, ``cp ~/unixplay/file_unixplay.txt .`` Note : Don't forget the dot ``.`` at the end! Remember, in Unix, the dot means the current directory. Also notice that we used full pathname of the original ( source ) file including the shortcut " ~ " for your home directory. The above command means copy the file ``file_unixplay.txt`` , which is in ``unixplay/`` (itself a sub-directory of your *home* directory) to the current directory ("``.``"), keeping the name the same. The effect of these commands is as follows: .. image:: imgs/filesys3.jpg :align: center | .. note:: You can *clone* a whole directory tree and its contents (i.e. including subdirectories) by using the "*Recursive*" copy switch, ``-r``, with ``cp``. Doing ``cp -r dir1 dir2`` will place a copy of ``dir1/`` *and ALL its contents and subdir-contents* in ``dir2/``. | Moving and Renaming Files and Directories ========================================= The ``mv`` command ------------------ The ``mv`` command can be used to "*move*" files *and whole directories* as well as to ``mv file1 file2`` moves, or really **renames**, file1 to file2 | To move a file from one place to another, you also use the ``mv`` command. This has the effect of moving rather than copying the file, so you end up with only one file rather than two. In this usage you would do: ``mv file1 dir1`` This would place ``file1`` in ``dir1``, and erase it from its current location. ---------------------------- Let's practice. Use ``mv`` to rename a file ``````````````````````````` We are now going to rename the file ``testfile3`` to ``testfile4``. First, change directories to your newdir/ directory (remember how?) . Then type ``l`` to see what files are there. Do you see ``testfile3`` that you made by copying ``testfile2`` above? Type ``mv testfile3 testfile4`` Now do a listing to see if ``testfile3`` is gone and replaced by ``testfile4`` (i.e. that ``testfile3`` was renamed to ``testfile4``). .. image:: imgs/filesys4.jpg :align: center Use ``mv`` to move a file to somewhere else ``````````````````````````````````````````` We will now move the file ``testfile4`` back into your ``unixplay/`` directory. Do a listing to verify that ``testfile4`` is still there in your ``newdir/`` directory. Now do ``mv testfile4 ../unixplay`` You can see in this command that the destination directory is up one level ( ``..`` ) and then down into the ``unixplay/`` directory. Verify that the move worked--that ``testfile4`` is not in the current directory ( ``newdir/`` ) and is now in ``unixplay/`` . .. image:: imgs/filesys5.jpg :align: center Use mv on a whole directory ``````````````````````````` Amazingly, you can ``mv`` a directory--either in the sense of renaming or moving it. ``cd`` to your home directory . Do a listing to see that the directories ``unixplay/`` and ``newdir/`` are there. Now do ``mv newdir dirTHREE`` Do you see that this is using **mv** to rename the directory? Verify that it worked. Was ``newdir/`` been renamed to ``dirTHREE/`` ? Still in you home dir, do ``mv dirTHREE unixplay`` This will move the newly renamed ``dirTHREE/`` into your ``unixplay/`` directory. .. image:: imgs/filesys3.jpg :align: center Verify that ``dirTHREE/'' is now a subdirectory of ``unixplay/``. There should also be a ``dirONE/`` and ``dirTWO/`` there as well. | Removing Files and Directories ============================== .. warning:: The ``rm`` and ``rmdir`` commands erases your files and dirs. Unix doesn't have a *Recycle Bin* where you can find the files you accidentally deleted. This Unix OS considers you to be God. If you say "*delete this file*, Unix complies with no wimpering; No "Do you really want to do that" warnings, or other protestations. So, be aware when you delete (``rm``) files and directories. rm and rmdir ------------ To delete (or remove) a file, use the ``rm`` command. As an example, we are going to create a copy of the ``pope.txt`` file then delete it. Do a listing inside your ``unixplay/`` directory Is the file ``file_unixplay.txt`` there? We don't need this one anymore. Do ``rm file_unixplay.txt`` List the directory's contents. The ``file_unixplay.txt`` should have been listed before you did the ``rm`` command, but then be gone in the second listing, since you ``rm``'ed it. You can use the ``rmdir`` command to remove a directory (make sure it is empty first). Try to remove the ``unixplay/`` directory. You will not be able to do so, since Unix will not let you remove a non-empty directory with ``rmdir`` (without taking extra steps). You have two options for removing a directory that has files in it (*assuming you don't need the files!!*). 1. You can ``cd`` into the directory, *delete all* the files by typing ``rm *``, then ``cd`` back up to the parent directory (``cd ..``), and typing ``rmdir dir``. 2. You can use the (*very long*) switch, ``--ignore-fail-on-non-empty``, as in ``rmdir --ignore-fail-on-non-empty dir``. 3. You can actually use: ``rm``, the same command you use for files. Just give it the ``-rf`` switch. These are **force** (``-f``) and **recursive** (``-r``). This will travel down the directory and its subdirs, removing everything in its path. .. warning:: This last way: ``rm -rf dir`` is convenient, but note that it is *very powerful*! If you cd to ``/`` (the top of the unix file system), and type ``rm -rf`` you will erase your *entire* disk, with no way to recover (unless you are making backups). That would probably be bad. Similarly, people have been known to do ``rm -rf *`` thinking they were down in a subdirectory and wanted to prune everything below. Then they realized that they were actually in their *home* directory. Usually they swear a lot when they realized what's happened. So--BEWARE. With great power comes great responsibility! Create a directory called ``tempstuff`` using ``mkdir``, ``cd`` into ``tempstuff/`` and do ``touch zap``. The ``touch`` command will create a file of zero size named ``zap`` in the ``tempstuff/`` directory. Now try to delete the ``tempstuff/'' directoty using either the ``rmdir`` or ``rm -rf`` commands. You might try the experiment with both commands. Manipulating the contents of files ================================== ``cat`` (Concatenate) --------------------- The command ``cat`` (for *concatenate* or to join ) can be used to display the contents of a file on the screen. Type: ``cat testfile`` This is fine since the file contains a small amount of text which all fits on the screen. If the file is bigger than will fit in an xterm window, we will use the **less** (pager) command, which you have already met before. You can also do ``cat file1 file2 file3``. This will concatenate (add together) all three files into one long file and spew it to the screen. Unless the file is small enough to fit whole on the screen, cat is most often used to pass the contents of one file to another program. Try it with the three files in the **unixplay/** dir. Do ``cat testfile anotherFile yetanother`` This should produce :: This is a text file, called testfile. This is another file. This is yet another file. The ``cat`` command printed the contents of each file, one after the other. ``less`` (Page through a file) ------------------------------ The command ``less`` displays the contents of a file onto the screen one page at a time, which is much more suitable for actually reading a file. In your ``unixplay/`` dir, you will find a a subdir called `dirONE`` which contains the file "``KublaKhan.txt``". ``cd`` to ``dirONE/``, then do ``less KublaKhan.txt`` Press the **SPACE-BAR** (or PgDn) if you want to see the next page, and "**b**" (or PgUp) to go back. Type "**q**" when you want to quit reading. As you can see, ``less`` is much better than ``cat`` for reading long files. ``less`` is a modern replacement to the original Unix paging file viewer called "``more``". The ``more`` program printed a page at a time, but offered little other functionality. It's a bit of Unix geek humor that the replacement for the ``more` program would be named "``less``". ``less`` offers many features for searching and moving. More on this below. You can see more of the features and how to use them by typing "**h**" while viewing a file with ``less`` (type "**q**" to "Quit" ``Help``).. | ``head`` (the first lines of a file) ------------------------------------ The ``head`` command writes the first *ten* lines of a file to the screen. First clear your screen (by typing ``clear``, of course!), then ``cd`` to your ``unixdir/`` directory. First find the file ``ints.dat`` and view it with ``less``. It contains the first 100 integers. Now type ``head ints.dat`` Then type ``head -5 ints.dat`` What difference did the -5 option make to the head command? This generalizes to ``-n``, where **n** is the number of lines you want to print. | ``tail`` (the last lines of a file) ----------------------------------- The ``tail`` command writes the *last* ten lines of a file to the screen. ``clear`` the screen and type ``tail ints.dat`` **Question**: How can you view the last 15 lines of the file? | Searching the contents of a file ================================ Simple searching using less --------------------------- Using ``less``, you can search though a text file for a keyword (or text pattern). For example, to search through ``KublaKhan.txt`` for the word **pleasure**, type ``less KublaKhan.txt`` (You should be able to find this file by now, even if you forgot where it is). Now, *while still viewing the file in* ``less``, type a *forward slash* (**/**) followed by the word you wish to search for, like this: ``/pleasure`` As you can see, less finds and highlights the keyword. Type the " **n**" key to search for the next occurrence of the word. | ``grep`` (search multiple files for a word) ------------------------------------------- This is a *REALLY* useful command. (don't ask why it is called *grep*. Ok, if you must know, the answer's `here `_) ``grep`` is one of many standard UNIX utilities. It searches files for specified words or patterns. First ``clear`` the screen, ``cd`` into ``unixplay/``, do a listing and find the file ``scifi_list.txt``. This is a (somewhat outdated) list of the top Science Fiction books of all time, in random order. Have a look with ``less``. Then type ``grep time scifi_list.txt`` As you can see, grep has printed out each line containg the word **time** (in color too!) *Or has it* ???? Try typing ``grep Time scifi_list.txt`` The ``grep`` command, by default, is case sensitive; it distinguishes between **time** and **Time**. To ignore upper/lower case distinctions, use the ``-i`` switch, i.e. type ``grep -i time scifi_list.txt`` Try it. To search for a phrase or pattern, you must enclose it in **single** quotes (the apostrophe symbol). For example to search for the phrase **Time Machine** , type ``grep 'Time Machine' scifi_list.txt`` Some of the other options of grep are: ``-v`` display only those lines that do NOT match the expression ``-n`` precede each matching line with the line number ``-c`` print only the total count of matched lines Try some of them and see the different results. Don't forget, you can use more than one option at a time. For example, the *number* of lines *without* the words **time** or **Time** is ``grep -ivc time scifi_list.txt`` | ``wc`` (word count) ------------------- A handy little utility is the ``wc`` command, short for *Word Count*. To do a word count on ``scifi_list.txt`` , type ``wc -w scifi_list.txt`` To find out how many lines the file has, type ``wc -l scifi_list.txt`` With no options ``wc scifi_list.txt`` wc prints: :: 100 746 4173 scifi_list.txt which shows the number of: *lines words bytes filename*. | Sort the contents of a file =========================== The file ``scifi_list.txt`` is nice; it contains the top 100 Science Fiction books. However, it would be nicer if they weren't in *random* order! No problem--use ``sort``! First have a look at the file using ``less``. The columns are :: 1995 53 Stephenson, Neal The Diamond Age 1956 36 Bester, Alfred The Stars My Destination 2000 89 Reynolds, Alastair Revelation Space ... or *Year Rank Author Title*. If you simply do ``sort scifi_list.txt`` you will order the file **alphabetically on the first column**: :: 1818 52 Shelley, Mary Frankenstein 1864 57 Verne, Jules Journey to the Center of the Earth 1870 32 Verne, Jules 20,000 Leagues Under the Sea 1884 92 Abbott, Edwin A Flatland 1889 86 Twain, Mark A Connecticut Yankee in KA's Court 1895 16 Wells, H G The Time Machine 1897 77 Wells, H G The Invisible Man 1898 18 Wells, H G The War of the Worlds ... Notice that "*alphabetical*" on the year, ends up ordering the years. This is because alphabetical order is **0123...89ABCD...XYZabcd...xyz**. Nice! Now we see the top books, ordered by year. How about sorting by *rank* (in **column two**)? Do: ``sort -k 2 scifi_list.txt`` which means "*sort alphabetically on column 2*". You should see: :: 1965 1 Herbert, Frank Dune 1968 10 Clarke, Arthur C 2001: A Space Odyssey 1974 100 Lem, Stanislaw The Cyberiad 1970 11 Niven, Larry Ringworld 1959 12 Heinlein, Robert A Starship Troopers 1968 13 Dick, Philip K Do Androids Dream of Electric Sheep? 1932 14 Huxley, Aldous Brave New World 1973 15 Clarke, Arthur C Rendezvous With Rama 1895 16 Wells, H G The Time Machine 1966 17 Heinlein, Robert A The Moon is a Harsh Mistress 1898 18 Wells, H G The War of the Worlds 1989 19 Simmons, Dan Hyperion 1985 2 Card, Orson Scott Ender's Game 1954 20 Clarke, Arthur C Childhood's End 1950 21 Bradbury, Ray The Martian Chronicles ... You can see that the books are ordered by the second column now, but the order is *wrong*!. It's alphabetical, but what we really wanted was "**numerical**" ordering. This time do: ``sort -k 2 -n scifi_list.txt`` This time we get it the way we wanted: :: 1965 1 Herbert, Frank Dune 1985 2 Card, Orson Scott Ender's Game 1951 3 Asimov, Isaac Foundation 1979 4 Adams, Douglas Hitch Hiker's Guide to the Galaxy 1949 5 Orwell, George 1984 1961 6 Heinlein, Robert A Stranger in a Strange Land 1954 7 Bradbury, Ray Fahrenheit 451 1984 8 Gibson, William Neuromancer 1950 9 Asimov, Isaac I, Robot 1968 10 Clarke, Arthur C 2001: A Space Odyssey 1970 11 Niven, Larry Ringworld 1959 12 Heinlein, Robert A Starship Troopers 1968 13 Dick, Philip K Do Androids Dream of Electric Sheep? 1932 14 Huxley, Aldous Brave New World 1973 15 Clarke, Arthur C Rendezvous With Rama ... We could also alphabetize the file on the author's names: ``sort -k 3 scifi_list.txt`` which gives us: :: 1884 92 Abbott, Edwin A Flatland 1979 4 Adams, Douglas Hitch Hiker's Guide to the Galaxy 1950 9 Asimov, Isaac I, Robot 1951 3 Asimov, Isaac Foundation 1954 29 Asimov, Isaac The Caves of Steel 1955 49 Asimov, Isaac The End Of Eternity 1972 43 Asimov, Isaac The Gods Themselves 1985 68 Atwood, Margaret The Handmaid's Tale 1988 56 Banks, Iain M Player Of Games 1990 65 Banks, Iain M Use of Weapons 1985 97 Bear, Greg Blood Music 1985 60 Bear, Greg Eon 1953 64 Bester, Alfred The Demolished Man 1956 36 Bester, Alfred The Stars My Destination ... Try: ``sort -k 3 -r scifi_list.txt`` What did the ``-r`` option do? ``sort`` is *VERY* handy! | Command Summary =============== +--------------------------+----------------------------------------------------+ | Command | Meaning | +==========================+====================================================+ | ``cp`` *file1 file2* | make a copy of file1 called file2 | +--------------------------+----------------------------------------------------+ | cp file1 dir | put a copy of file1 in dir | +--------------------------+----------------------------------------------------+ | cp f1 f2 f3 dir | copy files f1 f2 and f3 to dir | +--------------------------+----------------------------------------------------+ | cp fi\* dir | copy all files which match "fi"... to dir | +--------------------------+----------------------------------------------------+ | fin file | find file, by searching . and all subdirs | +--------------------------+----------------------------------------------------+ | fin fi\\* |find files matching "fi" (wildcard: use \\*, not \*)| +--------------------------+----------------------------------------------------+ | mv file1 file2 | move or rename file1 to file2 | +--------------------------+----------------------------------------------------+ | mv file1 dir | move file1 into dir | +--------------------------+----------------------------------------------------+ | rm file | remove a file or files | +--------------------------+----------------------------------------------------+ | rmdir dir | remove an empty directory | +--------------------------+----------------------------------------------------+ | rm -rf dir | forcibly delete directory, files, and subdirs | +--------------------------+----------------------------------------------------+ | cat file | print a file | +--------------------------+----------------------------------------------------+ | cat f1 f2 f3 | print a files f1, f2, and f3 all together | +--------------------------+----------------------------------------------------+ | less file | display a file a page at a time | +--------------------------+----------------------------------------------------+ | head file | print the first few lines of a file | +--------------------------+----------------------------------------------------+ | head -30 file | print first 30 lines of a file (works for any n) | +--------------------------+----------------------------------------------------+ | tail -n file | print the last n lines of a file | +--------------------------+----------------------------------------------------+ | grep 'pattern' file\* | check file(s) for 'pattern' | +--------------------------+----------------------------------------------------+ |grep -i 'pattern' file\* | check file(s)... ignore CASE of match | +--------------------------+----------------------------------------------------+ |grep -v 'pattern' file\* |check file(s)... report if DOES NOT contain pattern | +--------------------------+----------------------------------------------------+ |grep -n 'pattern' file\* | check file(s)... report LINE NUMBER of match | +--------------------------+----------------------------------------------------+ |grep -c 'pattern' file\* | check files(s)... report NUMBER OF MATCHING LINES | +--------------------------+----------------------------------------------------+ | wc file |count number of LINES / WORDS / CHARACTERS in file | +--------------------------+----------------------------------------------------+ | sort file |sort lines of a file, ALPHABETICALLY on 1st column | +--------------------------+----------------------------------------------------+ | sort -k N file | sort lines of a file, on N-th COLUMN | +--------------------------+----------------------------------------------------+ | sort -n -k N file |sort lines of a file, NUMERICALLY, on N-th column | +--------------------------+----------------------------------------------------+ | sort -r file | sort lines of a file, report in REVERSE order | +--------------------------+----------------------------------------------------+