Sunday, March 1, 2015

How to remove columns from CSV file based on column number using bash shell

Cite from :http://linuxconfig.org/how-to-remove-columns-from-csv-based-on-column-number-using-bash-shell

In the following example we are going to show how to remove columns from CSV file based on the column number. Consider a following command separated file containing 10 columns:
$ cat temp.csv 
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10
1,2,3,4,5,6,7,8,9,10

In the first example we are going to remove second column. The best tool for this job is cut command:
$ cut -d, -f2 --complement temp.csv 
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
1,3,4,5,6,7,8,9,10
Next, we will remove all columns in range 2-4 and 7,9:
$ cut -d, -f2-4,7-9 --complement temp.csv 
1,5,6,10
1,5,6,10
1,5,6,10
1,5,6,10
1,5,6,10
1,5,6,10
1,5,6,10
Remove the --complement option to remove columns 1,5,6,10:
$ cut -d, -f2-4,7-9 temp.csv 
2,3,4,7,8,9
2,3,4,7,8,9
2,3,4,7,8,9
2,3,4,7,8,9
2,3,4,7,8,9
2,3,4,7,8,9
2,3,4,7,8,9
In the last example we will remove columns 1,5,7 using bash variable:
$ remove='1,5,7'
$ cut -d, -f$remove --complement temp.csv 
2,3,4,6,8,9,10
2,3,4,6,8,9,10
2,3,4,6,8,9,10
2,3,4,6,8,9,10
2,3,4,6,8,9,10
2,3,4,6,8,9,10
2,3,4,6,8,9,10

No comments:

Post a Comment