Batch multiple csv files to utf-8 encoding

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
shantanu97
Posts: 3
Joined: 24 Apr 2022 00:45

Batch multiple csv files to utf-8 encoding

#1 Post by shantanu97 » 24 Apr 2022 00:57

I have a bunch of CSV files with different encoding. How can I batch convert files in a directory for their encoding with a cmd in windows??? My final encoding is UTF-8. Ideally, what I want in the script to loop through every CSV file in the directory, and convert it to into a UTF-8.
Some files are in the directory are ANSI,UTF-16,UTF-8-BOM,UTF-16 LE BOM
ANSI --> utf-8
utf-16-->utf-8
utf-8-bom--->utf-8
utf-16-le bom --->utf-8

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Batch multiple csv files to utf-8 encoding

#2 Post by aGerman » 24 Apr 2022 10:29

I wrote a little utility to serve those kind of tasks.
CONVERTCP
Ideally you should know the original encoding of the file and feed the utility with the right ID for every file. "Charset-guessing" is extremely error prone. So, first write the converted files into another folder (instead of overwriting the original files) in order to see how much crap you get.

Code: Select all

md "convertcp_out"
for %%i in (*.csv) do convertcp.exe ? 65001 /i "%%~i" /o "convertcp_out\%%~i"
Steffen

shantanu97
Posts: 3
Joined: 24 Apr 2022 00:45

Re: Batch multiple csv files to utf-8 encoding

#3 Post by shantanu97 » 28 Jun 2022 22:31

Hi Steffen,

I am facing a problem when I am passing input folder locations in the below command.
My input folder is present on the desktop and my output folder is on documents.
How to change the below code if the input folder and output folder are in different locations. In the input folder there are a bunch of CSV files there.

Code: Select all

md "C:\Users\xyz\Documents\Test\UTF8Files"
for %%i in (C:\Users\xyz\Desktop\UTF8\Input\*.csv) do convertcp.exe ? 65001 /i "%%~i" /o "C:\Users\xyz\Documents\Test\UTF8Files\%%~i"
pause

aGerman
Expert
Posts: 4654
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Batch multiple csv files to utf-8 encoding

#4 Post by aGerman » 29 Jun 2022 09:13

viewtopic.php?f=3&t=7570&p=67297#p67297
Answered over there?

Steffen

Post Reply