Bill Bellon -- SSEC/UW-Madison

renfiles: Rename Files utility (BETA)

Download | Man page | Example | Filenaming Template | Date formatting | Future improvements | More Examples

renfiles is a script for renaming a set of files. It's useful if you have a set of files that all have the same filenaming convention, i.e.,
a1.08012.USA1.143.1000m.jpg a1.08013.USA1.143.1000m.jpg a1.08014.USA1.143.1000m.jpg
and you want to rename the files in a consistent way.

The basic idea of renfiles is that you give it a file list (i.e., *.jpg) and a file naming template and it will take that template and apply it to each file in the file list you give it, copying or moving the files in the process as you specify. The file naming template allows you to select different parts of a filename and rearrange them as well as transform them, such as converting from one date format to another or zero padding numbers.

Download


Save the file to your computer and then upload to a UNIX machine that has PHP installed on it. It's set to run like a UNIX shell script (but uses the PHP parser instead of korn or bash shell). You may have to change the first line in the script to point at the correct location for the PHP executable on your UNIX machine. Sometimes you can just do a
which php
to find the PHP executable on your UNIX machine.

Man page

Rename / copy / move files using a string splitting syntax and sprintf and date formatting capabilities Usage: rename-files -h rename-files -t newFileNameTemplate [-e] [-c|m] [-d dir] -f filelist rename-files -tf templateFile [-e] [-c|m] [-d dir] -f filelist rename-files -t newFileNameTemplate [-e] [-c|m] [-d dir] -fl filelist_file rename-files -tf templateFile [-e] [-c|m] [-d dir] -fl filelist_file -f filelist or -fl filelist_file MUST be the last command line parameters! Or your computer will explode! -h help -t newFileNameTemplate template for new filenames -tf templateFile text file that contains newFileNameTemplate on one line -e echo only (default) -c copy filelist to dir -m move filelist to dir -d dir diretory where to copy or move filelist (default: current dir) -f filelist file containing a list of files to copy / move - one filename per line -fl filelist_file file containing a list of files to copy / move - one filename per line

IMPORTANT! either use -e or don't specify -c or -m when first using this - the -e (which is the default) will just echo the results of the command and won't do anything - that way you can test a command first and not royally FUBAR yourself.

Example:

Convert the YYYYMMDD in a set of filenames to YYYYDOY (Day Of Year) format:
%ls *jpg a1.20080112.USA1.143.1000m.jpg a1.20080113.USA1.143.1000m.jpg a1.20080114.USA1.143.1000m.jpg a1.20080115.USA1.143.1000m.jpg %./renfiles -m -t 'a1.{[.|1]!Yz!}.USA1.143.1000m.jpg' -f *.jpg %ls *.jpg a1.2008012.USA1.143.1000m.jpg a1.2008013.USA1.143.1000m.jpg a1.2008014.USA1.143.1000m.jpg a1.2008015.USA1.143.1000m.jpg

The above command is applying a filenaming template (specified by -t 'a1.{[.|1]!Yz!}.USA1.143.1000m.jpg') to each file in the file list (specified via -f *.jpg). The -m tells renfiles to use the mv (move) command to rename the files.

Filenaming Template Explained:

To understand the above example, you need to understand how the filenaming template works. Let's break the above example down.

a1.{[.|1]!Yz!}.USA1.143.1000m.jpg

This is a filenaming template and all such templates are composed of:

  • strings - example: a1. and .USA1.143.1000m.jpg
  • template variables - example: anything between {}, i.e. {[.|1]!Yz!}

template variables

Template variables ({}) are composed of:
  • selectors - [delim|index] or [|n,m]
  • operators - !! or ""

Selectors

Selectors ([]) allow you to select any part of a filename, either by splitting the filename up into parts and selecting which part you want (i.e., [delim|index]), or by using the equivalent of a substring() method (i.e., [|n,m]).

Examples:
[.|1] splits
a1.20080115.USA1.143.1000m.jpg
into this array:
0 => a1 1 => 20080115 2 => USA1 3 => 143 4 => 1000m 5 => jpg
and selects index 1, namely
20080115.

See the help page on PHP split for more help on this concept.

The other way we could select
20080115
out of
a1.20080115.USA1.143.1000m.jpg
would be to use the substring feature:
[|3,8]
This says to grab 8 characters out of a1.20080115.USA1.143.1000m.jpg starting at character position 3.
See the PHP substr function for more help on this feature.

Selectors can also be chained together in two different ways:
to select the YY (i.e., 08) out of
a1.20080115.USA1.143.1000m.jpg
we could use
[.|1][|2,2]
The following diagram illustrates how this works:

The other way to chain selectors together is using : - instead of feeding the output of the previous selector into the input of the next selector, using : concatenates the results of each selector into one string, which is then operated on by a final "" or !! at the end of a template variable - see me for more help on this (this doc is getting too long as it is)

Operators

Once you have selected a part of a filename, you can then transform or operate on it using either "" or !!.

"format" runs the string you have selected through the PHP sprintf() function (used to format strings) using the format you specify

!format! runs the string you have selected through the PHP date function (used to format dates) using the format you specify.

Here is a list of date format examples that renfiles understands and can reformat (keep in mind you can use the other operators and selectors to convert your date into a format that renfiles understands):

  • 2004013, 1999263, 2009365 ... (i.e., YEAR followed by Day of Year)
  • 15Jan2004, 15january2004, 15.Jan.2004, 15-jan-2004 ...
  • 2004Jan15, 2004january15, 2004.Jan.15, 2004-jan-15 ...
  • 20040115, 2004.01.15, 2004-1-15 ...

Date formatting

Here are some brief notes on this (more will be added later). This script can reformat dates in a filename (using the !! operator). Here are some examples of the date formats that renfiles currently understands and can reformat:

Year day_of_year: 2004001, 2007118 ....
Year month_number day: 20040115, 2004.01.15, 2004-1-15 ....
Day month_name year: 15Jan2004, 15january2004, 15.Jan.2004, 15-jan-2004 ....
Year month_name day: 2004Jan15, 2004january15, 2004.Jan.15, 2004-jan-15....


I have plans to add more date formats that it can understand. If renfiles can understand a date format then you can use it to reformat that date (in a filename) to almost any other format you want.

Future improvements

Here is a list of possible future improvements (contact me if you want something not on this list):
  • Regular expressions
  • Maybe changing the syntax and how renfiles works so that strings must be enclosed in something (like "string") so that if the filenaming template only specifies a template variable (i.e., {}) then renfiles just selects that part of the filename and operates on it - i.e., so that you can just specify a part of the filename you want to select and operate on rather than having to specify the entire filename via the template. Example - if you look at the first example above, you could rewrite it as

    %./renfiles -m -t '{[.|1]!Yz!}' -f *.jpg

    instead of

    %./renfiles -m -t 'a1.{[.|1]!Yz!}.USA1.143.1000m.jpg' -f *.jpg

    Or maybe just add a command line parameter to tell renfiles to act this way rather than the way it currently does (which is that you specify the entire filename in the filenaming template).

More Examples:

Convert filenames like this
a1.20080115.USA1.143.1000m.jpg --> modis_15.Jan.08_USA01_00143_1000m.jpg a1.20081009.USA2.11442.1000m.jpg --> modis_09.Oct.08_USA02_11442_1000m.jpg a1.19990526.USA10.4398.1000m.jpg --> modis_26.May.99_USA10_04398_1000m.jpg

you could do this

./renfiles -t 'modis_{[.|1]!d.M.y!}_USA{[.|2][|3]"%02d"}_{[.|3]"%05d"}_1000m.jpg' -f a1.*.jpg