05-28-2017 05:16 AM
I want to start Trex on 8 ports and while starting the trex, i am observing below error:
Number of ports found: 8
wait 10 sec ..........
port : 0
------------
link : link : Link Up - speed 1000 Mbps - full-duplex
promiscuous : 0
port : 1
------------
link : link : Link Up - speed 10 Mbps - full-duplex
promiscuous : 0
port : 2
------------
link : link : Link Up - speed 1000 Mbps - full-duplex
promiscuous : 0
port : 3
------------
link : link : Link Up - speed 1000 Mbps - full-duplex
promiscuous : 0
port : 4
------------
link : link : Link Up - speed 1000 Mbps - full-duplex
promiscuous : 0
port : 5
------------
link : link : Link Up - speed 100 Mbps - full-duplex
promiscuous : 0
port : 6
------------
link : link : Link Up - speed 1000 Mbps - full-duplex
promiscuous : 0
port : 7
------------
link : link : Link Up - speed 100 Mbps - full-duplex
promiscuous : 0
-------------------------------
RX core uses TX queue number 1 on all ports
core, c-port, c-queue, s-port, s-queue, lat-queue
------------------------------------------
1 0 0 1 0 2
2 2 0 3 0 2
3 4 0 5 0 2
4 6 0 7 0 2
-------------------------------
number of ports : 8
max cores for 2 ports : 1
max queue per port : 3
RING: Cannot reserve memory
ERROR there is not enough huge-pages memory in your system
EAL: Error - exiting with code: 1
Cause: Cannot init nodes mbuf pool nodes-2
07-21-2017 08:04 AM
Difference in scenario 1 and scenario 2:
Scenario 1: User 1 uses 2 ports with default zmq ports, able to start the server in first instance
User 2 uses 6 ports with user-defined zmq ports, prefix(different username). Able to start the server in second instance
Scenario 2: User 1 uses 2 ports with default zmq ports, able to start the server in first instance
User 2 using more than 6 ports be it 8 or more ports with user-defined zmq ports, prefix(different username). Unable to start the server in second instance with options specified mbuf, no-scapy
[root@trex-server v2.27]# free utility
total used free shared buff/cache available
Mem: 65861408 9017368 56410488 32676 433552 56472612
Swap: 32964604 0 32964604
[root@trex-server v2.27]# free -h
total used free shared buff/cache available
Mem: 62G 8.6G 53G 31M 423M 53G
Swap: 31G 0B 31G
[root@trex-server v2.27]# /proc/meminfo
-bash: /proc/meminfo: Permission denied
[root@trex-server v2.27]# cat /proc/meminfo
MemTotal: 65861408 kB
MemFree: 56409692 kB
MemAvailable: 56472652 kB
Buffers: 33620 kB
Cached: 268444 kB
SwapCached: 0 kB
Active: 196152 kB
Inactive: 163880 kB
Active(anon): 94008 kB
Inactive(anon): 952 kB
Active(file): 102144 kB
Inactive(file): 162928 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 32964604 kB
SwapFree: 32964604 kB
Dirty: 372 kB
Writeback: 0 kB
AnonPages: 62316 kB
Mapped: 137768 kB
Shmem: 32676 kB
Slab: 132288 kB
SReclaimable: 46284 kB
SUnreclaim: 86004 kB
KernelStack: 8304 kB
PageTables: 4856 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 61701004 kB
Committed_AS: 368112 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 429904 kB
VmallocChunk: 34325686432 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 4096
HugePages_Free: 3072
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 120364 kB
DirectMap2M: 6057984 kB
DirectMap1G: 62914560 kB
[root@trex-server v2.27]#
07-21-2017 08:50 AM
Understood now. Running two TRexs instances is not simple. Making it running in optimal way it even more complex..
Our script run good for single instance.
1) Please send the config file for user 1) and user 2)
Make sure they have different cores assigned.
2) try to reduce mbuf in both instances to 0.2 (using --mbuf-factor)
You should have at least 8GB (tlb) + 2GB (heap) in your system
Hanoh
Sent from my iPhone
07-22-2017 03:43 AM
Now, I'm able to run the server with two instances from user1 and user2 using mbuf argument set 0.2.
Could you please tell me how to specify the cores accordingly to ports being used in yaml file?
Config information of user1 and user2 yaml files,
User1:
[root@trex-server v2.27]# cat cfg/asp.yaml
- port_limit : 2
version : 2
interfaces : ["86:00.0","86:00.1"]
prefix : server_iot
port_info :
- src_mac : '00:00:00:71:00:01'
- src_mac : '00:00:00:71:00:02'
User2:
[root@trex-server v2.27]# cat cfg/Trex.yaml
- port_limit : 10
version : 2
interfaces : ["02:00.0","02:00.1","02:00.2","02:00.3","07:00.0","07:00.1","07:00.2","07:00.3","86:00.2","86:00.3"]
# interfaces : ["02:00.0","02:00.1","02:00.2","02:00.3","07:00.0","07:00.1","07:00.2","07:00.3"]
prefix : server_wipro
zmq_pub_port : 5001
zmq_rpc_port : 5002
port_info :
- default_gw : '2.2.2.1'
src_mac : '00:00:00:71:00:11'
ip : '2.2.2.2'
- default_gw : '2.2.2.2'
src_mac : '00:00:00:71:00:12'
ip : '2.2.2.1'
- default_gw : '3.2.2.1'
src_mac : '00:00:00:71:00:13'
ip : '3.2.2.2'
- default_gw : '3.2.2.2'
src_mac : '00:00:00:71:00:14'
ip : '3.2.2.1'
- default_gw : '4.2.2.1'
src_mac : '00:00:00:71:00:15'
ip : '4.2.2.2'
- default_gw : '4.2.2.2'
src_mac : '00:00:00:71:00:16'
ip : '4.2.2.1'
- default_gw : '5.2.2.2'
src_mac : '00:00:00:71:00:17'
ip : '5.2.2.1'
- default_gw : '5.2.2.2'
src_mac : '00:00:00:71:00:18'
ip : '5.2.2.1'
- default_gw : '6.2.2.2'
src_mac : '00:00:00:71:00:19'
ip : '6.2.2.1'
- default_gw : '6.2.2.2'
src_mac : '00:00:00:71:00:20'
ip : '6.2.2.1'
# - default_gw : '7.2.2.2'
# src_mac : '00:00:91:00:00:01'
# ip : '7.2.2.1'
# - default_gw : '7.2.2.2'
# src_mac : '00:00:91:00:00:02'
# ip : '7.2.2.1'
[root@trex-server v2.27]#
07-23-2017 01:20 AM
Hi,
Easiest way to do it is using script, look at the options here:
sudo ./dpdk_setup_ports.py --help
You can run it first time with usual options to create config similar to your first one.
You will see section with cores config.
Then, for second config file, you can specify cores to exclude ( --ce argument to the script ) which are going to be used in first instance of TRex.
08-17-2023 03:57 AM
It appears that you're encountering an issue while trying to start Trex with 8 ports. The error message indicates a memory reservation problem related to huge-pages. Here's a breakdown of what's happening and how you might address it:
When starting Trex, you've successfully detected 8 ports and their respective link status and configurations. However, when attempting to initialize the RX core and set up memory pools, a specific error arises:
RING: Cannot reserve memory
ERROR there is not enough huge-pages memory in your system
EAL: Error - exiting with code: 1
Cause: Cannot init nodes mbuf pool nodes-2
This error suggests that the system is unable to reserve the required memory for the huge-pages, leading to the initialization failure. Huge-pages are a mechanism that provides larger memory pages to improve memory management efficiency.
To address this issue:
Check System Configuration: Ensure that your system has sufficient memory resources, including available huge-page memory, to accommodate the needs of running Trex with 8 ports.
Adjust Huge-Page Settings: You may need to adjust the huge-page settings on your system to allocate more memory for Trex. This can usually be done through your operating system's configuration or settings.
Reduce Port Configuration: As a temporary measure, you could try starting Trex with fewer ports to see if the issue is related to resource constraints. If it starts successfully with fewer ports, gradually increase the number of ports while monitoring memory usage.
Resource Optimization: Review other resource-intensive processes running on your system that could be competing for memory. Temporarily stopping or optimizing these processes might free up memory for Trex.
Consult Documentation: Check the Trex documentation or community forums for any specific recommendations or solutions related to this memory allocation issue.
Remember, addressing memory-related issues often involves a combination of system-level adjustments, resource management, and potentially seeking guidance from relevant technical resources.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide