UNIX Archives - AlekZ' Scratchpad

One-liner: How to scan a network for up & down hosts

Another network scanner.

For example, you want to find possible free IP-addresses in the 10.0.10.1 – 10.0.10.1 range:

for i in {1..10}; do (ip="10.0.10.$i"; up=`ping -n -q -W3 -c1 $ip | fgrep transmitted | sed -e 's/.*100%.*/down/' -e 's/.*0%.*/up/'`;name=`host $ip | awk '!/NXDOMAIN/{print $NF}'`; echo $ip $up $name)  2>/dev/null & done | sort -n -t. -k4

Result:

10.0.10.1 down
10.0.10.2 down
10.0.10.3 up host1.
10.0.10.4 down
10.0.10.5 down myhost2.internal.domain.
10.0.10.6 up nas3.
10.0.10.7 down laptop.
10.0.10.8 down
10.0.10.9 up switch2.
10.0.10.10 down printer.

One-liner: how to change file extensions according to their type

What if you have PNG or WEBP files, saved with .jpg extension? Here’s a one liner to rename them accordingly. You need mediainfo package:

sudo apt-get install mediainfo

Imagine we have file1.jpg in the current directory:

% ls
file1.jpg
% file file1.jpg
file1.jpg: RIFF (little-endian) data, Web/P image, VP8 encoding, 800x800, Scaling: [none]x[none], YUV color, decoders should clamp

No problem just rename it! But what if we have a hundred?

for i in * 
do 
  oldext=`echo "$i" | sed -e 's/^.*\.\([A-Za-z][A-Za-z]*\)$/\1/'` 
  echo "Old ext=$oldext" 
  newext=`mediainfo "$i" | egrep "^Format[[:space:]]+:" | sort -u | tr '[A-Z]' '[a-z]' | sed -e 's/.* \([a-z][a-z]*\)/\1/' -e 's/jpeg/jpg/'` 
  echo "New ext=$newext" 

  oldname=`echo $i | sed -e 's/\.'$oldext'$/.'$oldext'/'` 
  newname=`echo $i | sed -e 's/\.'$oldext'$/.'$newext'/'` 

  echo old=$oldname 
  new=$newname 
  mv -i $oldname $newname 

done

The result:

Old ext=jpg
New ext=webp
old=file1.jpg new=file1.webp

% ls
file1.webp

One-liner: How to remove spaces in file names

Spaces in file names are nasty and might prevent various script from working properly. Here’s a one-liner to replace spaces with underscore for all files in the current directory:

for i in *
do
   mv "$i" ${i// /_}
done

One-liner: How to check positional numbers of elements in the CheckPoint logs

CheckPoint log entries are divided by semi-colons and can have … many … fields. How to quickly check the positional number of a particular field in a particular log entry? Here’s a quick AWK one-liner (in AWK the “0” element is the whole line):

$ echo '315918;1Jan2019;0:03:30;fe80::d123:3aaa:fe80:fb73;ff02::1;ipv6-icmp;;accept;;;;10.1.2.26;log;;eth1.123;inbound;VPN-1 & FireWall-1;;f-firewall001;Network;0;;;;;;;;;Implied rule;;;Neighbor Advertisement;136;0;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;' | \ 
awk -v RS=\; '{print NR,$0}'
1 315918
2 1Jan2019
3 0:03:30
4 fe80::d123:3aaa:fe80:fb73
5 ff02::1
6 ipv6-icmp
7 
8 accept
9 
10 
11 
12 10.1.2.26
13 log
14 
15 eth1.123
16 inbound
17 VPN-1 & FireWall-1
18 
19 f-firewall001
20 Network
21 0
22 
23 
24 
25 
26 
27 
28 
29 
30 Implied rule
31 
32 
33 Neighbor Advertisement
34 136
35 0
36 
37 
...
92

A bit longer alternative variant:

awk -F\; '{ for (i=1;i<=NF;i++) {print i,$i}}'

To number the field names (provided the logs are converted to TXT and gzipped):

$ zcat 2019-01-01_025249_2308.log.txt.gz | head -1 |  awk -v RS=\; '{ print NR,$0}'
1 num
2 date
3 time
4 src
5 dst
6 proto
7 service
8 action
9 xlatesrc
10 xlatedst
11 peer gateway
12 orig
13 type
14 alert
15 i/f_name
16 i/f_dir
17 product
18 log_sys_message
19 origin_id
20 ProductFamily
21 rule
22 rule_uid
23 rule_name
24 service_id
25 NAT_rulenum
26 NAT_addtnl_rulenum
27 s_port
28 xlatedport
29 xlatesport
30 message_info
31 inzone
32 outzone
33 ICMP
34 ICMP Type
35 ICMP Code
36 TCP packet out of state
37 tcp_flags
38 scheme:
39 methods:
40 encryption failure:
41 partner
42 community
43 fw_subproduct
44 vpn_feature_name
45 srckeyid
46 dstkeyid
47 IKE:
48 CookieI
49 CookieR
50 msgid
51 IKE notification:
52 Certificate DN:
53 IKE IDs:
54 user
55 rule_guid
56 hit
57 policy
58 first_hit_time
59 last_hit_time
60 log_id
61 message
62 ip_id
63 ip_len
64 ip_offset
65 fragments_dropped
66 during_sec
67 fw_message
68 reject_category
69 DCE-RPC Interface UUID
70 Log delay
71 description
72 status
73 version
74 comment
75 update_service
76 Protection Name
77 Severity
78 Confidence Level
79 protection_id
80 SmartDefense Profile
81 Performance Impact
82 Industry Reference
83 Protection Type
84 detected port
85 protocol
86 Attack Info
87 attack
88 FollowUp
89 Log ID
90 spi
91 encryption fail reason:
92 rpc_prog

One-liner: How to convert CheckPoint firewall logs

To make the log format predictable, create /etc/fw/conf/logexport.ini with the following

For R70 (Secuplat):

[Fields_Info]
included_fields=num,date,time,src,dst,proto,service,action,xlatesrc,xlatedst,peer gateway,<REST_OF_FIELDS>

For R77 (GAIA):

[Fields_Info]
included_fields=date,time,src,dst,proto,service,action,xlatesrc,xlatedst,peer gateway,<REST_OF_FIELDS>

Create a directory for the converted logs:

mkdir /var/log/2019.txt

Run the following command to convert all logs, for example, for January 2019:

 
for i in $FWDIR/log/2019-01-*.log; do echo $i; fwm logexport -n -p -i $i |  gzip -c - > /var/log/2019.txt/$i.txt.gz; done

One-liner: how to do a ping sweep in bash

Ping sweep from 10.0.1.1 to 10.0.1.31:

 for i in {1..31}; do ping -c 1 10.0.1.$i | fgrep ttl & done 2>/dev/null | sed -e 's/^.*from //' -e 's/:.*$//' | sort -n -t. -k4

Result:

10.0.1.1
10.0.1.2
10.0.1.3
10.0.1.7
10.0.1.9
10.0.1.10
10.0.1.12
10.0.1.13
10.0.1.14
10.0.1.15
10.0.1.16
10.0.1.17
10.0.1.18
10.0.1.19
10.0.1.20

One-liner: how to search media files for file and track names

Let’s find all media files with file or track names containing the word “brain” recursively beginning from the current directory:

find . -type f -exec mediainfo '{}' \+ | tr -s ' ' | tr -s '      ' | sed -e 's/ : /:/' | awk -F: 'function aprt () {printf("%s | %s | %s\n",a["Complete name"], a["Duration"], a["Track name"]);a["Complete name"]="";a["Duration"]="";a["Track name"]="";} {a[$1]=$2} /^General/ {aprt()} END {aprt()}' | fgrep -i brain

Result (file name | duration | track name):

./Jazz Side of the Moon/8 Brain Damage.flac | 6 min 20 s | 
./Pink Floyd - TDSOTM - (CP35-3017 Japanese Black Triangle)/8 Brain Damage.flac | 3 min 50 s | Brain Damage
./The Dark Side of The Moon/09.flac | 3 min 51 s | Brain Damage
./Vocomotion - Dark Side of the Moon/08 - Vocomotion - Brain Damage.mp3 | 3 min 53 s | Brain Damage

The “one-liner” above could have been simpler and faster, if there were no spaces in the file names.

One-liner: how to convert CheckPoint netconf.C routes to Gaia/Clish commands

Provided all route metrics are zeroes:

cat /etc/sysconfig/netconf.C | tr '(' ' '| tr ')' ' ' | tr '"' ' '| tr ':' ' ' | tr '\t' ' '| tr -s ' '| sed -e 's/^ //' | sed 's/routes//' | awk '/route/ {printf("set static-route ");} /dest/ {printf("%s ",$2);} /via/ {printf("nexthop gateway address %s ",$2);} /metric/ {print "on"}'

The result:

set static-route 10.13.198.160/27 nexthop gateway address 10.12.12.1 on
set static-route 10.13.198.192/27 nexthop gateway address 10.12.12.1 on
set static-route 192.168.112.0/24 nexthop gateway address 10.12.12.1 on
set static-route 192.168.113.0/24 nexthop gateway address 10.12.12.1 on
set static-route 192.168.114.0/24 nexthop gateway address 10.12.12.1 on
set static-route 192.168.115.0/24 nexthop gateway address 10.12.12.1 on
set static-route default nexthop gateway address 10.0.0.1 on

One-liner: how to check file growth in real time

Not quite a one-liner, but it can be copy&pasted in a terminal window 😉

bash
TIME=10
FILE=file.log
function fsize {
	SIZE=`ls -l $FILE | awk '{print $5}'`
}
fsize
S1=$SIZE
while true
do
	sleep $TIME
	fsize
	S2=$SIZE
	(( DIFF=($S2-$S1)/$TIME ))
	echo "($S2-$S1)/$TIME = $DIFF"
	S1=$S2
done

The script above will check the “file.log” size every 10 seconds and output the previous size, the new size and bytes per second:

(30117062-30085517)/10 = 3154
(30148404-30117062)/10 = 3134
(30179199-30148404)/10 = 3079
^C
^D

How to start Cisco ASDM from command-line (Windows, UNIX)

The following command can be used to start Cisco ASDM from command-line on Windows (without ASDM installation) or UNIX. Java must be locally installed:

javaws https:///admin/public/asdm.jnlp