netmiko errors out during 'archive download-sw /overwrite /imageonly'

[romanroma]

I am currently using netmiko to automate switch stack upgrades. However, i am running into the following error when my script runs

Traceback (most recent call last):
  File "./main-0.1.py", line 65, in <module>
    main()
  File "./main-0.1.py", line 63, in main
    upgrade(ios,ip,logFile)
  File "./main-0.1.py", line 40, in upgrade
    connection.send_command(stg)
  File "/usr/local/lib/python3.6/site-packages/netmiko/utilities.py", line 429, in wrapper_decorator
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1525, in send_command
    search_pattern
OSError: Search pattern never detected in send_command: c2960x\-tan2\-01\#

I am looping through a flat text config file, that has IP addresses in it. What is interesting, I am not sure why I am seeing "OSError: Search pattern never detected in send_command: c2960x\-tan2\-01\#" in the output, which is a mangled regex of the switch name. The script then executes the following command

connection.enable
connection.send_command(archive download-sw /overwrite /imageonly tftp://192.168.10.100/c2960x-universalk9-tar.152-7.E7.tar

I snipped some of the boiler plate code out and showed only the related command. However, what the interesting part is, the script errors out on my utility server, but the process completes correctly on the switch. I can even see the socket and session for the tftp server in use.

Due to the latter, I am not sure where to even begin to start to troubleshoot the issue, since the script barfs, and does not progress to the next IP in the config file. Maybe there is some timer or something that I have to set to null or none?

For an experiment, I was able to get the process done with Tcl/Expect, which works, but I had to set a global timer in the script to '-1', so the screen or session didn't time out while the upgrade took place. But I am not sure if it is related.

[jasoblac]

This describes working around the timeout

https://stackoverflow.com/questions/68797284/netmiko-oserror-search-pattern-never-detected-in-send-command

[romanroma]

Hi jasoblac,

I am not fully understanding all the timers, programming and scripting are not my strong points. However, I see there is a number of timers or delay factors that can be set. Do you have any recommendations or more options for discussion, since I fully do not understand the impact of each, nor when to use which function/mechanism.

So, from what I can understand, 'netmiko' is waiting for the prompt again; however, it times out due to the file upload time. Do you know of a global, just keep waiting until the script can process the remaining commands? I already have the script ending with the disconnect function, so in practice the script, once it moves forward will disconnect.

Thank you

[jasoblac]

in the device: section of your script try this

device = {
    "host": "xxxxxxxx",
    "device_type": "cisco_nxos",
    "username": "admin",
    "password": "xxxxxxxx",
    "fast_cli": False,  # Notice the item here   <---------------------------------- THIS
    "secret": "",  # Enable password (If applicable)
}

There is also this setting (it's all in the link I gave you)

send_command("show whatever", delay_factor=2) 

[romanroma]

I already have the 'fast_cli' in my device statement. In addition, I tried the 'delay_factor=2', but did not work. do you know of any global setting that just turns off all delay factors or settings? For example in Expect, 'set timeout -1' is what is required to work around the long time, and suppressed cli output of the 'archive download' command. Due to the latter, I suspect something similar is required for netmiko; however, I am unsure.

[romanroma]

Looks like the "send_command_timing" function worked; however, it is a little kludgy, since no output is generated, and the script just ran. But looking at Linux 'losf' output and seeing the 'upgrade' directory on switches, would indicate the update started. However, since the script completed and exited without errors before the upgrade could finish, there is no capability for error checking with 'sho boot' like I do in my Expect script.

It is looking as if Expect is still a better option at this point for me; however, I would love to standardize one tool-chain even though I am not a python fan.

[romanroma]

Upgrade did take place; however, there is no screen output like there is with an Expect script. In addition, the script exits success, but you would not know this. After sometime, the time it usually takes to upgrade the switch stack, and the snmp cpu alarms due to using the 'archive download' command the switches were upgraded.

Have others scripted switch stack upgrades with nemiko or other similar tools?