If your Power over Ethernet (PoE) device(s) connected to an EX 4300-48p switch suddenly lose power unexpectedly after some time working just fine, you may be experiencing PoE controller software issue.
First of all, verify if you are not over your PoE power budget (this could be sometimes the case when you have a LOT of devices connected – this kind of situation occurs only if you have a lot of type 2 high powered PoE (IEEE 802.3at) devices connected to it) :
show chassis power-budget-statistics
You should see a similar output like this :
user@HOSTNAME> show chassis power-budget-statistics
fpc 0:
-------------------------------------------------------------------------
PSU 0 ) : 0 W Offline
PSU 1 (JPSU-1100-AC-AFO-A ) : 1100 W Online
Power redundancy configuration : N+0
Total power supplied by all online PSUs : 1100 W
Base power reserved : 175 W
Non-PoE power being consumed : 140 W
Total power allocated for PoE : 925 W
Total PoE power consumed : 0 W
Total PoE power remaining : 925 W
{master:0}
You should look at the “Total PoE power consumed”. If you are equal or tight to the limit, it might be just your problem, you should disconnect a few devices/relocate them on another switch and see if there is any improvement.
In this particular situation, the output above was taken immediately after the “PoE outage” occurred. All devices were connected properly, but no power was injected by the switch and consumed by the devices. Down/Up the port(s) had no effect.
It then came to mind to have a look at the “PoE controller”
show poe controller
Here is an example of the output of this command :
user@HOSTNAME> show poe controller
Controller Maximum Power Guard Management Status Lldp
index power consumption band Priority
0** 925.00W 66.44W 0W Class AT_MODE Disabled
**New PoE software upgrade available.
Use 'request system firmware upgrade poe fpc-slot <slot>'
This procedure will take around 10 minutes (recommended to be performed during maintenance)
{master:0}
As we can see, there is a notification that a new version of the PoE software is available. In my case, that was the issue, most likely a bug in the current version that I was running.
NOTE : It may be a good idea to validate first if you are running the recommended JunOS version by JTAC. If not, upgrading that first is recommended.
NOTICE : Before you continue, this upgrade need to be performed within a maintenance window, a minimum of 10 – 15 minutes will be required, as in addition of upgrading the PoE software, the switch will automatically reboot once completed. The switch may become unstable while performing the upgrade.
In order to perform the PoE software upgrade, do the following command :
request system firmware upgrade poe fpc-slot 0
NOTE : Please note that if you are running a virtual-chassis, the command will need to be performed for all unit and change the “fpc-slot” number matching the other units in the VC.
You will now notice the following output once you’ve issued the PoE firmware upgrade :
user@HOSTNAME> request system firmware upgrade poe fpc-slot 0
Firmware upgrade initiated. Poe Upgrade takes about 10 minutes
Use 'show poe controller' to get the download status,
Please Reboot the system after Upgrade is complete.
{master:0}
user@HOSTNAME> request system firmware upgrade poe fpc-slot 0
Message from syslogd@HOSTNAME at Mar 20 08:40:54 ...
HOSTNAME pfex: SCHED: Thread 53 (PoE Firmware Download) aborted, hogged 13341 ms
Message from syslogd@HOSTNAME at Mar 20 08:41:01 ...
HOSTNAME pfex: SCHED: Thread 53 (PoE Firmware Download) aborted, hogged 6818 ms
[...]
Message from syslogd@HOSTNAME at Mar 20 08:46:51 ...
HOSTNAME pfex: SCHED: Thread 50 (TNPC CM) ran for 1806 ms without yielding
Message from syslogd@HOSTNAME at Mar 20 08:46:51 ...
HOSTNAME pfex: Scheduler Oinker
Message from syslogd@HOSTNAME at Mar 20 08:46:51 ...
HOSTNAME pfex: Frame 00: sp = 0x05e98840, pc = 0x0184d424
Message from syslogd@HOSTNAME at Mar 20 08:46:51 ...
HOSTNAME pfex: Frame 01: sp = 0x05e98858, pc = 0x0181082c
[...]
*** System shutdown message from user@HOSTNAME ***
System going down in 30 seconds
*** FINAL System shutdown message from user@HOSTNAME ***
System going down IMMEDIATELY
Once the switch(es) completed their upgrade and rebooted, the PoE injection should be stable and steady.
[This article may also apply to EX2200, EX3200, EX2300, EX3300, EX3400-p chassis as well]