Custom shell commands with wildcard characters in Go CI

Photo by hannah joshua / Unsplash

For your build pipeline, ThoughtWork’s GO let you define tasks to invoke a shell command or script. This is extremely useful for tasks such as ssh’ing or scp’ing files to deployment servers.

For this post, we will take example of SCP.

You can setup task to scp a single file as shown in following screenshot:

For single file uploads, this work perfectly fine. But if you want to upload multiple files (for example, all zip files in the folder), you would like to specify it with shell wildcard characters like * or ?.

Unfortunately, this does not work. We get following error with above settings:

dist/*.zip: No such file or directory

The problem here is Go internally puts all parameters inside single quotes(‘) while executing command. So, even if you intended the command to run as

scp dist/*.zip user1@my_server.com:/home/user1/deployment

the actual command which gets executed is

scp 'dist/*.zip' 'user1@my_server.com:/home/user1/deployment'

As the parameter is inside single quote, shell will not try to interpolate the wildcard character * in the command.

To overcome this limitation, you can use bash’s -c switch. This bash switch lets you specify a string which will executed in a shell’s context.

After specifying above settings, you can achieve the intended results.

Shirish Padalkar

Shirish Padalkar

Pune, India