pipe binary mode

[lazna]

Trying to contruct command which download and unzip set of files in following manner:

Code: Select all

curl.exe -o- "https://host.com/release/file.zip" | tar.exe -xf - -C "C:\work\target_dir"

but some archive files missing in extracted structure. Suspect Windows Defender, but its log display nothing. What could be wrong? Does windows pipe support binary mode at all?

Now using following workaround with temp file, but I did like to avoid it.

Code: Select all

curl.exe -o "%temp%\file.zip" "https://host.com/release/file.zip" && tar.exe -xf "%temp%\file.zip" -C "C:\work\target_dir"

[penpen]

Without any sample data, we can only guess.

The pipe exclusively works in binary mode and till now, i never saw it fail. Any encoding is done either by the command line interpreter, or by the programs on each end of the pipe.
In my experience the most common error is that the program on the sink side (here: "tar.exe") interprets a special character, such as for example the SUB character (U+001A), as an end of stream indicator and stops reading data from the pipe.
But it might also be caused by program one the source side (here: "curl.exe"), or might be a codepage issue, or ... .
If that issue is causes by the sink or source program, there is not much you could do (except reporting that bug with sample data to the author of the buggy program and hope for an update).


You might check, whether or not the data on the sink side is correct by saving it to a file and compare it with the downloaded file:

Code: Select all

@echo off
curl.exe -o- "https://host.com/release/file.zip" | >"%temp%\file.piped.zip" powershell.exe -c "[Console]::OpenStandardInput().CopyTo([Console]::OpenStandardOutput())"
:: download file
curl.exe -o "%temp%\file.zip" "https://host.com/release/file.zip"
:: binary compare both files
fc /b "%temp%\file.zip" "%temp%\file.piped.zip"

penpen