DISCLAIMER: This is my first time trying to deploy any kind of fiber network connection via Ubuntu. Please pardon my ignorance.
ISSUE:
I'm currently trying to get a Moloch (https://molo.ch/) cluster online and connect to a Gigamon packet capture broker. The connection is Gigamon directly to the Moloch Capture server (10Gb fiber). The server itself is a Dell R740xd with a Dell/Intel X710 controller. Lights blink on both the Gigamon and Dell server side, but I can't get the interface to act like I think it's supposed to/can't tcpdump any traffic from the connected port. Below are some outputs in an attempt to help. If you have any clue on what I can do to provide more information or what I have missed, please let me know. The operating system is 16.04.5. Thanks!
ip a - List of network interfaces. eno1 is the fiber, enp23sof3 is the online ethernet
ifconfig - Shows the ethernet port, but not the fiber
ethtool -i eno1 - "Proof" that diver is installed
ethtool eno1 - Ethtool not displaying supported ports as FIBER and other things
ethtool -S eno1 - Apparently receiving bytes, but not packets?
lshw -class net
lspci -vv - More data just in case :)
ISSUE:
I'm currently trying to get a Moloch (https://molo.ch/) cluster online and connect to a Gigamon packet capture broker. The connection is Gigamon directly to the Moloch Capture server (10Gb fiber). The server itself is a Dell R740xd with a Dell/Intel X710 controller. Lights blink on both the Gigamon and Dell server side, but I can't get the interface to act like I think it's supposed to/can't tcpdump any traffic from the connected port. Below are some outputs in an attempt to help. If you have any clue on what I can do to provide more information or what I have missed, please let me know. The operating system is 16.04.5. Thanks!
ip a - List of network interfaces. eno1 is the fiber, enp23sof3 is the online ethernet
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noop portid 246e96b38ba0 state DOWN group default qlen 1000
link/ether brd
3: enp23s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether brd
4: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid 246e96b38ba2 state DOWN group default qlen 1000
link/ether brd
5: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid 246e96b38ba4 state DOWN group default qlen 1000
link/ether brd
6: enp23s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether brd
7: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid 246e96b38ba6 state DOWN group default qlen 1000
link/ether brd
8: enp23s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether brd
9: enp23s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether brd
inet XXX.XXX.XXX.XXX/22 brd XXX.XXX.XXX.XXX scope global enp23s0f3
valid_lft forever preferred_lft forever
inet6 XXXX::XXXX:XXXX:XXXX:XXXX/64 scope link
valid_lft forever preferred_lft foreverCode:
enp23s0f3 Link encap:Ethernet HWaddr XX:XX:XX:XX:XX:XX
inet addr:XXX.XXX.XXX.XXX Bcast:XXX.XXX.XXX.XXX Mask:XXX.XXX.XXX.0
inet6 addr: XXXX::XXXX:XXXX:XXXX:XXXX/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17022962 errors:0 dropped:22004 overruns:0 frame:0
TX packets:122134 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1376501711 (1.3 GB) TX bytes:15786185 (15.7 MB)
Memory:af100000-af1fffff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:275722 errors:0 dropped:0 overruns:0 frame:0
TX packets:275722 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20405652 (20.4 MB) TX bytes:20405652 (20.4 MB)Code:
driver: i40e
version: 2.1.14-k
firmware-version: 6.00 0x800034eb 18.3.6
expansion-rom-version:
bus-info: 0000:18:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yesCode:
Settings for eno1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: g
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: noCode:
NIC statistics:
rx_packets: 0
tx_packets: 0
rx_bytes: 0
tx_bytes: 0
rx_errors: 0
tx_errors: 0
rx_dropped: 0
tx_dropped: 0
collisions: 0
rx_length_errors: 0
rx_crc_errors: 0
rx_unicast: 0
tx_unicast: 0
rx_multicast: 0
tx_multicast: 0
rx_broadcast: 0
tx_broadcast: 0
rx_unknown_protocol: 0
tx_linearize: 0
tx_force_wb: 0
rx_alloc_fail: 0
rx_pg_alloc_fail: 0
tx-0.tx_packets: 0
tx-0.tx_bytes: 0
rx-0.rx_packets: 0
rx-0.rx_bytes: 0
tx-1.tx_packets: 0
tx-1.tx_bytes: 0
rx-1.rx_packets: 0
rx-1.rx_bytes: 0
tx-2.tx_packets: 0
tx-2.tx_bytes: 0
rx-2.rx_packets: 0
rx-2.rx_bytes: 0
tx-3.tx_packets: 0
tx-3.tx_bytes: 0
rx-3.rx_packets: 0
rx-3.rx_bytes: 0
tx-4.tx_packets: 0
tx-4.tx_bytes: 0
rx-4.rx_packets: 0
rx-4.rx_bytes: 0
tx-5.tx_packets: 0
tx-5.tx_bytes: 0
rx-5.rx_packets: 0
rx-5.rx_bytes: 0
tx-6.tx_packets: 0
tx-6.tx_bytes: 0
rx-6.rx_packets: 0
rx-6.rx_bytes: 0
tx-7.tx_packets: 0
tx-7.tx_bytes: 0
rx-7.rx_packets: 0
rx-7.rx_bytes: 0
tx-8.tx_packets: 0
tx-8.tx_bytes: 0
rx-8.rx_packets: 0
rx-8.rx_bytes: 0
tx-9.tx_packets: 0
tx-9.tx_bytes: 0
rx-9.rx_packets: 0
rx-9.rx_bytes: 0
tx-10.tx_packets: 0
tx-10.tx_bytes: 0
rx-10.rx_packets: 0
rx-10.rx_bytes: 0
tx-11.tx_packets: 0
tx-11.tx_bytes: 0
rx-11.rx_packets: 0
rx-11.rx_bytes: 0
tx-12.tx_packets: 0
tx-12.tx_bytes: 0
rx-12.rx_packets: 0
rx-12.rx_bytes: 0
tx-13.tx_packets: 0
tx-13.tx_bytes: 0
rx-13.rx_packets: 0
rx-13.rx_bytes: 0
tx-14.tx_packets: 0
tx-14.tx_bytes: 0
rx-14.rx_packets: 0
rx-14.rx_bytes: 0
tx-15.tx_packets: 0
tx-15.tx_bytes: 0
rx-15.rx_packets: 0
rx-15.rx_bytes: 0
port.rx_bytes: 156202083579
port.tx_bytes: 0
port.rx_unicast: 186491052
port.tx_unicast: 0
port.rx_multicast: 29035
port.tx_multicast: 0
port.rx_broadcast: 5052
port.tx_broadcast: 0
port.tx_errors: 0
port.rx_dropped: 0
port.tx_dropped_link_down: 0
port.rx_crc_errors: 108
port.illegal_bytes: 95
port.mac_local_faults: 1190
port.mac_remote_faults: 0
port.tx_timeout: 0
port.rx_csum_bad: 0
port.rx_length_errors: 0
port.link_xon_rx: 0
port.link_xoff_rx: 0
port.link_xon_tx: 0
port.link_xoff_tx: 0
port.rx_size_64: 29867714
port.rx_size_127: 25831366
port.rx_size_255: 21973336
port.rx_size_511: 6870033
port.rx_size_1023: 4686691
port.rx_size_1522: 97296002
port.rx_size_big: 0
port.tx_size_64: 0
port.tx_size_127: 0
port.tx_size_255: 0
port.tx_size_511: 0
port.tx_size_1023: 0
port.tx_size_1522: 0
port.tx_size_big: 0
port.rx_undersize: 0
port.rx_fragments: 1
port.rx_oversize: 0
port.rx_jabber: 94
port.VF_admin_queue_requests: 0
port.arq_overflows: 0
port.rx_hwtstamp_cleared: 0
port.tx_hwtstamp_skipped: 0
port.fdir_flush_cnt: 0
port.fdir_atr_match: 0
port.fdir_atr_tunnel_match: 0
port.fdir_atr_status: 1
port.fdir_sb_match: 0
port.fdir_sb_status: 1
port.tx_lpi_status: 0
port.rx_lpi_status: 0
port.tx_lpi_count: 0
port.rx_lpi_count: 0
port.tx_priority_0_xon: 0
port.tx_priority_0_xoff: 0
port.tx_priority_1_xon: 0
port.tx_priority_1_xoff: 0
port.tx_priority_2_xon: 0
port.tx_priority_2_xoff: 0
port.tx_priority_3_xon: 0
port.tx_priority_3_xoff: 0
port.tx_priority_4_xon: 0
port.tx_priority_4_xoff: 0
port.tx_priority_5_xon: 0
port.tx_priority_5_xoff: 0
port.tx_priority_6_xon: 0
port.tx_priority_6_xoff: 0
port.tx_priority_7_xon: 0
port.tx_priority_7_xoff: 0
port.rx_priority_0_xon: 0
port.rx_priority_0_xoff: 0
port.rx_priority_1_xon: 0
port.rx_priority_1_xoff: 0
port.rx_priority_2_xon: 0
port.rx_priority_2_xoff: 0
port.rx_priority_3_xon: 0
port.rx_priority_3_xoff: 0
port.rx_priority_4_xon: 0
port.rx_priority_4_xoff: 0
port.rx_priority_5_xon: 0
port.rx_priority_5_xoff: 0
port.rx_priority_6_xon: 0
port.rx_priority_6_xoff: 0
port.rx_priority_7_xon: 0
port.rx_priority_7_xoff: 0
port.rx_priority_0_xon_2_xoff: 0
port.rx_priority_1_xon_2_xoff: 0
port.rx_priority_2_xon_2_xoff: 0
port.rx_priority_3_xon_2_xoff: 0
port.rx_priority_4_xon_2_xoff: 0
port.rx_priority_5_xon_2_xoff: 0
port.rx_priority_6_xon_2_xoff: 0
port.rx_priority_7_xon_2_xoff: 0Code:
*-network:0 DISABLED description: Ethernet interface
product: Ethernet Controller X710 for 10GbE SFP+
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:18:00.0
logical name: eno1
version: 01
serial: XX:XX:XX:XX:XX:XX
width: 64 bits
clock: 33MHz
capabilities: bus_master cap_list rom ethernet physical autonegotiation
configuration: autonegotiation=off broadcast=yes driver=i40e driverversion=2.1.14-k firmware=6.00 0x800034eb 18.3.6 latency=0 link=no multicast=yes promiscuous=yes
resources: irq:51 memory:ae000000-aeffffff memory:af018000-af01ffff memory:af700000-af77ffffCode:
18:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
DeviceName: NIC1
Subsystem: Dell Ethernet 10G 4P X710 SFP+ rNDC
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 51
Region 0: Memory at ae000000 (64-bit, prefetchable) [size=16M]
Region 3: Memory at af018000 (64-bit, prefetchable) [size=32K]
Expansion ROM at af700000 [disabled] [size=512K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=129 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00001000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L1, Exit Latency L0s <2us, L1 <16us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
Capabilities: [e0] Vital Product Data
Product Name: X710 10GbE Controller
Read-only fields:
[V0] Vendor specific: FFV18.3.6
[PN] Part number: 68M95
[MN] Manufacture ID: 31 30 32 38
[V1] Vendor specific: DSV1028VPDR.VER2.0
[V3] Vendor specific: DTINIC
[V4] Vendor specific: DCM1001FFFFFF2101FFFFFF3201FFFFFF4301FFFFFF1402FFFFFF2502FFFFFF3602FFFFFF4702FFFFFF1803FFFFFF2903FFFFFF3A03FFFFFF4B03FFFFFF1C04FFFFFF2D04FFFFFF3E04FFFFFF4F04FFFFFF
[V5] Vendor specific: NPY4
[V6] Vendor specific: PMT7
[V7] Vendor specific: NMVIntel Corp
[V8] Vendor specific: L1D0
[RV] Reserved: checksum good, 1 byte(s) reserved
Read/write fields:
[Y1] System specific: CCF1
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt+ UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn+ ChkCap+ ChkEn+
Capabilities: [140 v1] Device Serial Number XX-XX-XX-XX-XX-XX-XX-XX
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Capabilities: [1d0 v1] #19
Kernel driver in use: i40e
Kernel modules: i40e