DOS BATCH SCRIPTS

1. Had to frequently pull files from a server. The file set  used to be in following format.

abc123_file1.txt
def23145_file2.txt
r4rtyyy_file3.txt

The task was to remove the prefix from names and put the file set back. It was too mechanical, so I wrote following script to do the same.


ECHO OFF

 SETLOCAL EnableDelayedExpansion

 FOR %%I IN (*) DO (
   REM ECHO %%I
   SET  xname=%%I
 
   for /f "tokens=1,2 delims=_" %%a in ("%%I") do (
     REM ECHO %%a
     REM ECHO %%b  
     REM ECHO !xname!
     SET xname=!xname:%%a_=!
     REM ECHO !xname!
     SET ENDSWITH=!xname:~-4!
     REM ECHO !ENDSWITH!
     IF "!ENDSWITH!"==".csv"  REN %%I  !xname!
     IF "!ENDSWITH!"=="done"  REN %%I  !xname!
   )
)


2.  I faced  a situation where I needed to upload files to server at some intervals and not in one go. I was looking for a way to write script. This is what I did.  I was using winSCP  to upload the file.  I came up with following batch file.

ECHO OFF

SETLOCAL EnableDelayedExpansion

FOR %%I IN (*) DO (
   REM ECHO %%I
   SET  xname=%%I
   SET ENDSWITH=!xname:~-5!
   ECHO !xname!
   copy wscp.txt wscp2.txt
   IF "!ENDSWITH!"==".sgnl"  (
     ECHO Put D:\Users\gbairwa\Documents\reprocess\poc\!xname!  /dest/path/>> wscp2.txt
     ECHO close>>wscp2.txt
     ECHO exit>>wscp2.txt
   )
   type wscp2.txt
   # pause
   "C:\Program Files (x86)\WinSCP\winscp.com" /ini=nul /script=wscp2.txt
   # Below command to introduce some wait/sleep
   ping 192.0.2.2 -n 1 -w 100000 > nul
)

wsp.txt
--------
open -hostkey="ssh-rsa 2048 f8:c3:55:ef:df:45:fc:48:89:48:ac:e8:d5:70:1f:8d" userid:password@host.domain
option transfer binary

Comments

Post a Comment

Popular posts from this blog

SQL

Joins - Not always very obvious ( Following example uses Hive ). create table x_temp ( name string, p_date int  ) insert into x_temp(name,p_date) values ('x1',20181025),('x2', 20181026), ('x3',20181027) create table y_temp ( name string, s_date int  ) insert into y_temp(name,s_date) values ('x1',20181025),('x2', 20181026), ('x3',20181027) create table z_temp ( name string, p_date int, s_date int ) insert into z_temp(name,p_date,s_date) values ('z1','20181025','20181025') select x.*, y.* from x_temp x, y_temp y, z_temp z  where x.p_date != z.p_date and  y.s_date != z.s_date) generates 4 records... x.name,x.p_date,y.name,y.s_date x2, 20181026, x2, 20181026 x3, 20181027, x2, 20181026 x2, 20181026, x3, 20181027 x3, 20181027, x3, 20181027 select x.*, y.* from x_temp x, y_temp y, z_temp z  where x.p_date != z.p_date or  y.s_date != z.s_date) generates 8 records x.name,x.p_date,y.
Read more

Analytics

Types of Analytics ( Notes from AWS Course )  Descriptive - What happened ?  hindsight  Diagnostic - Why did it  happen?  hindsight & insight  Predictive  - What will happen ? insight & foresight  Prescriptive - What should I do ? foresight  Cognitive & AI  - What are recommended actions ? foresight & hypothesis input  Popular BI Tools:  https://www.cio.com/article/3284415/10-bi-tools-for-data-visualization.html
Read more

HIVE

1. When doing multi table joins, except the last ( right most ) table, other tables are buffered in the reducers. So to have less memory footprint on reducers, largest tables should be the last one as it is not buffered but streamed to reducers.   For Impala , it is different. 2.  You can filter the unwanted records within the join itself. SELECT  a.*,   b. * FROM a LEFT OUTER JOIN b ON ( a.key=b.key  AND  b.name= 'Tim'   AND a .name= 'Kim' ) 3. Joins are left associative. So execution happens from left to right and the result of joins is  fed into next join operation. 4. To find the value of variable     set hive.execution.engine 5. To find values of all variables set  6. If you want to pass different hive configuration settings in hive shell, you can use --hiveconf. For example hive --hiveconf hivi.optimize.sort.dynamic.partition=true --hiveconf hive.exec.dynamic.partition.mode=nonstrict 7.  In case you accidentally drop the p
Read more