Nodachisoft Nodachisoft logo, Katana Sword Icon
  
@Amaji✎ Update November-9-2021

How to use xxd command to replace binary data such as line feed codes with Linux commands

How to use xxd command to replace binary data such as line feed codes with Linux commands

Summury

This is a reminder of how to remove and replace line feed codes in text and replace binary data on Linux. It can also be run on Windows by Windows users by installing cygwin or WSL2, and If you install git, you can easily convert it with the commands available as a subset of git-bash.

Specifically, you can use the xxd command to replace the binary data with hexadecimal text and then do the conversion.

Prep (for Windows users and git-bash)

This is a reminder to preparetion (pass) for git user. If you already have xxd command available, skip it.

Once you have the PATH env, you can run the Linux commands installed with git-bash from any command prompt at your Windows.

From the win key menu, select git-bash Application and launch.

Once git-bash is up, type where xxd to know where is the xxd executable.

check xxd path

Now that we know that the location is C:\Program Files\Git\usr\bin\, we can add this path to the PATH env in System Preferences. Add this path to your PATH in System Preferences.

Cannot be processed by sed or tr command

The Linux commands sed and tr can be used to read and process text data line by line.

The xxd command can handle them well.

Sample: Deleting a newline code

This is an example of how to remove the line feed code from a UTF-8 text file whose line feed code is CR LF.

Let's output test_oneline.txt, in which the newline code is removed from the text.txt file as shown below.

text.txt
squirrel
watermelon
kangaroo

Actually, let's remove the newline code from the text.txt file.

code_example
xxd -p -c 68719476736 test.txt | sed s/0d0a//g | xxd -p -r > test_oneline.txt

The resulting file will look like this, with the newlines removed!

text_oneline.txt
squirrelwatermelonkangaroo

What it does

The xxd command can output the contents of a file as hexadecimal text data, or vice versa, back to binary data. The procedure, with the newline code removed, is as follows:

  1. read the file with xxd and output it as hexadecimal text data.
  2. use the sed command to delete the character string "0d0a" when the newline code (CR LF) is expressed in hexadecimal.
  3. convert the deleted hexadecimal text data into binary data with xxd

My note for using xxd command

The options for the xxd command are as follows

Option Summury
-p Standard output in hexadecimal text
-c Specifies the number of hexadecimal digits to output per line. The default is 16; if the number exceeds 16 columns, the line will be broken accordingly. For example, if you specify "68719476736", you can output up to 64GB without line feeds.
-r Read hexadecimal numbers in text and output as binary to standard output

Websites and references that were very helpful.

Page History

date modification
2021-Dec-15 Added the use of git-bash to the xxd command
 
 
Message sent

Thank you for your message.

Something error has occured!

Sorry. The Error has occurred.We apologize for the inconvenience.Please try again in a few minutes or contact us via DM below.

Twitter:@NodachiSoft_eng
Name:
 
Replay To:
 
Message:
 
Back
Check the content!

Send the following information to us. If you are happy with your submission, please click "Send". If you want to modify it, please click "Back".

Name:
 
Reply To:
 
Message:
 
Enter a confirmation key to make sure that you are not operating from a Robot.
Confirmation Key is 95
Back
 / 
Go to Confirmation
Entry fields
Go to Confirmation

There are 0 articles that may be relevant!

Table Of Contents
How to use xxd command to replace binary data such as line feed codes with Linux commands
How to use xxd command to replace binary data such as line feed codes with Linux commands
Summury
Summury
Prep (for Windows users and git-bash)
Prep (for Windows users and git-bash)
Cannot be processed by sed or tr command
Cannot be processed by sed or tr command
Sample: Deleting a newline code
Sample: Deleting a newline code
What it does
What it does
My note for using xxd command
My note for using xxd command
Websites and references that were very helpful.
Websites and references that were very helpful.
Page History
Page History
Nodachisoft © 2021