Back to Contents

Linux Driver Software: Broadcom NetXtreme™ Gigabit Ethernet Adapter User's Guide

These notes describe the installation and configuration of the Linux driver for the Broadcom NetXtreme™ Gigabit Ethernet Adapter and include the following information:

InstallationPatching PCI FilesNetwork InstallationModule ParametersStatisticsBASP Configuration


NOTE – The current version of the driver has been tested on Redhat 7.1 and 7.2 (beta) Linux distributions for i386 and IA64 (kernel version 2.4.x).

Installation

Overview

The driver is released in the following packaging formats: source RPM, compressed tar, and binary RPM formats. The file names for the two packages are bcm5700-<version>.src.rpm and bcm5700-<version>.tar.gz respectively. Identical files to build the driver are included in both packages.

NOTE – The following procedures require that you be logged on as "root" or equivalent.

Installing the Source RPM Package

To install the Source RPM Package:

  1. Install RPM source package as follows:

rpm -ivh bcm5700-<version>.src.rpm

  1. Change directory to the RPM path and build the binary driver for your kernel as follows:

    cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..}
    rpm -bb SPECS/bcm5700.spec

    Note that the RPM path is different for different Linux distributions.

  1. Install the newly built package::

rpm -ivh RPMS/i386/bcm5700-<version>.i386.rpm

Note that the --force option is needed if installing on Redhat 7.1, which already contains an older version of the driver.

The driver will be installed in /lib/modules/... and the man page will also be installed.

  1. Load the driver:

insmod bcm5700

Building Driver From TAR File

  1. Create a directory and extract the files:

    tar xvzf bcm5700-<version>.tar.gz

  2. Build the driver bcm5700.o as a loadable module for both uniprocessor and multiprocessor systems:

    make

  3. Test the driver by loading it:

    insmod bcm5700.o

  4. Complete the installation:

    make install

Unloading and Removing Driver

  1. Bring down all BCM570x interfaces using ifconfig or ifdown:

    e.g. ifdown eth0

  2. Unload the driver:

    rmmod bcm5700

  3. If the driver was installed using RPM, it can be removed using:

    rpm -e bcm5700-<version>

  4. If the driver was installed from the tar archive, the driver can be manually deleted as follows:

    Redhat 7.1/7.2
    rm /lib/modules/<kernel version>/kernel/drivers/net/bcm/bcm5700.o



Patching PCI Files (Optional)

To use Redhat's kudzu hardware detection utility, a number of files containing PCI vendor and device information need to be patched with information on the BCM5700/01 series network adapaters. Patch files for Redhat 7.1 (pci-rh71.patch) and Redhat 7.2 (pci-rh72.patch) are included. Apply the patch by doing the following:

patch -N -p1 -d /usr < pci-rh70.patch
- or -
patch -N -p1 -d /usr < pci-rh71.patch

Run kudzu:

kudzu


Network Installation

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver diskette that contain the bcm5700 driver is needed for Redhat 7.1. The driver diskette's images for Redhat 7.1 are located in network_install/rh71/dd.img respectively. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from Redhat's website.

To create the driver diskette do the following:

dd if=dd.img of=/dev/fd0H1440.


Module Parameters

Optional parameters for the driver can be supplied as command line arguments to the insmod command. These parameters take the form:

<parameter>=value[,value,...]

where the multiple values for the same parameter are for multiple NICs installed in the system.

Note that default values will be used when invalid values are selected. All the parameters are listed below.


Statistics

Detailed statistics and configuration information can be viewed in the file /proc/net/nicinfo/eth#.info. Where # is the number of a physical device.


Broadcom Advanced Server Program (BASP)

BASP Overview

BASP is a kernel module designed for Linux 2.2.x and 2.4.x kernels that provides load-balancing, fault-tolerance, and VLAN features. These features are provided by creating teams that consist of multiple NIC interfaces. A team can consist of 1 to 8 NIC interfaces and each interface can be designated primary or standby. All primary NIC interfaces in a team will participate in Load-balancing operations by sending and receiving a portion of the total traffic. Standby interfaces will take over in the event that all primary interfaces have lost their links. VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual device is created for each VLAN added.

BASP supports Smart Load-balance, SLB and Generic trunking. In SLB mode, all the NIC drivers must support Broadcom NIC Extension (NICE). In this release, several NIC drivers patched with NICE are included. SLB mode works with all Ethernet switches without configuring the switch ports to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound directions. The Generic trunking mode does not require NICE and can work with any NIC, however, it requires the Ethernet switch to support the technology and be properly configured. This mode is protocol-independent and all traffic should be load-balanced and fault-tolerant.

Limitations

BASP supports Red Hat 6.2, 7.0, 7.1 and 7.2 Beta. The following installation procedures work with these distributions. BASP has also been tested on SuSE 7.1 & 7.2, Caldera 2.4 & 3.1, Turbo Linux 6.5 and Mandrake 8.0. Minor modification to the Makefile may be required if problems are experienced when compiling BASP on other I386 Linux Distributions.

BASP also supports Red Hat Linux 7.1 for IA-64 architecture and has been tested on Redhat Linux 7.2 Beta for IA-64.

VLANs are only supported by Broadcom NetXtreme Gigabit Ethernet. As opposed to VLANs support in other platforms, e.g. Windows and Netware, VLANs are not supported by Alteon Acenic driver (acenic.c).

BASP Installation

For users of Redhat 6.2, 7.0, 7.1 and 7.2, please follow instructions in
"Install - RPM package" section.

For users of other Linux I386 distributions, please follow instructions in
"Install - TAR archive" section.

For users of Redhat 7.1 or 7.2 Beta for IA-64, please follow instructions in
"Install - RPM package" section.

For users of other Linux IA-64 distributions, please follow instructions in
"Install - TAR archive" section

Installing TAR Archive

BASP for Linux is shipped in mixed forms, where the platform and kernel specific files are in source code, and the core file is in object form. Four packages are shipped in this release; two tar archives and two RPM packages:

"basplnx-{version}.i386.tgz" is the tar archive for i386 platform, and

"basplnx-{version}.ia64.tgz" is the tar archive for IA-64 platform.

To uncompress and expand the tar archive, run

% tar xvfz basplnx-{version}.{arch}.tgz

To Install the TAR archive, perform the following steps:

  1. To build kernel module, "basp.o",

    % make

  2. To create device file and to copy files,

    % make install

NOTE – If when building the BASP module, an error message regarding "invalid IO or IRQ parameters" are observed, you will need to configure the major device number for BASP configuration. The default major number is 250 and it can be changed by modifying "Makefile".
  1. To update the module reference,

    % depmod -a

  2. To load the driver,

    % insmod basp

  3. Follow the Configuration section to set up the teams.

Installing RPM Package

To install the Source RPM Package:

  1. To install the RPM source package, run:

% rpm -ivh basplnx-{version}.src.{arch}.rpm

  1. Change directory to the RPM path and build the binary driver for the kernel:

% cd /usr/src/redhat
% rpm -bb SPECS/basplnx.spec

Note that the RPM path is different for different Linux distributions.

  1. Install the newly built package:

% rpm -ivh RPMS/i386/basplnx-{version}.{arch}.rpm

The driver and other required files will be installed.

  1. Load the driver:

% insmod basp

  1. Follow the instructions in the Configuration section to set up the teams.

Files

Makefile makefile
baspcfg precompiled configuration utility
bcmtype.h commonly use type header file
blf.c BASP module entry points
blf.h ioctl interface
blfcore.h core interface
blfcore.o precompiled core object
blfopt.h automatically generated header file from Make
blfver.h version header file
nicext.h NICE header file
pal.c platform abstraction implementation
pal.h header for platform abstraction
release.txt this file
nice-2.2.16 contains NICE enabled driver sources code
scripts contains sample scripts
scripts/basp init script, goes to /etc/rc.d/init.d
scripts/baspteam   start/stop script, goes to /etc/basp
scripts/baspif   start/stop network, i/f, goes to /etc/basp
scripts/team-sample sample script of SLB team with three NICs
scripts/team-gec sample script of GEC team with three NICs
basp.4
man page
baspcfg.8 man page for baspcfg utility

BASP Configuration

The BASP distribution includes a utility program and several scripts for team configuration. These scripts are customized for Redhat distribtutions and automate the loading of basp and the initialization of the virtual interface at boot time.

For users of other Linux distributions, please follow instructions in the "baspcfg" section.

Configuration - Redhat Distributions

To configure a team perform the following steps:

  1. Copy a configuration script from "/etc/basp/samples" directory to
    "/etc/basp" directory. Note that the configuration script must be prefixed with "team-".
  1. Modify the configuration script to:

    a) change the team type
    b) add/delete physical network interfaces
    c) add/delete virtual network interfaces

The syntax of the configuration script can be found below.

  1. After modifying sample script for custom configurations, start teaming as follows:

% /etc/rc.d/init.d/basp start

NOTE – The configuration is saved and will automatically start on subsequent reboots.
  1. For Redhat Linuz, run "netcfg" to configure the IP addresses of the virtual network interfaces.
NOTE 1 – For Redhat 7.2, where netcfg is no longer supported, you will need to manually create the interface configuration file. The path and name of the file is:
/etc/sysconfig/network-scripts/ifcfg-sw0, Where sw0 is your virtual interface name.

Sample configuration file for virtual interface sw0:

DEVICE=sw0
BOOTPROTO=static
IPADDR=192.168.0.1
NETMASK=255.255.255.0
NETWORK=192.168.0.0
BROADCAST=192.168.0.255
ONBOOT=yes

Please read your distribution specific documentation on configuring your interface.

NOTE 2 – Starting the team is only required for the first time, and the teams will be started automatically in subsequent reboots.
NOTE 3 – Forming multiple teams is possible by copying the sample files into /etc/basp/team-<name>" and modifying this file as described in the sample file.
NOTE 4– In order to create more that one virtual interface (VLAN) for each team, refer to the respective description section in the sample files.

 

Configuration using baspcfg

The baspcfg command line tool is used to configure the BASP teams, add and remove NICs, and add and remove a virtual devices. This tool, may be used to configure your own startup script files. Please read your distribution specific documentation for more information on your startup procedure. The following is an example of the usage of this tool.

baspcfg v1.2.0 - Broadcom Advanced Server Program Configuration Utility
Copyright (c) 2000-2001 Broadcom Corporation. All rights reserved.

usage: baspcfg <commands>

commands

addteam <tid> <type> <tname> -- create a team

delteam <tid> -- delete a team

addva <tid> <vlan_id> <vname> [macaddr] -- add a virtual adapter to a team

delva <tid> <vlan_id> -- del a virtual adapter from a team bind

bind <tid> <role> <device> -- bind a physical adapter to a team unbind

unbind <tid> <device> -- unbind a physical adapter from a team

show <tid> -- display team configurations

where

tid -- A unique ID for each team, starting from 0

type -- Team type: 0=SLB, 1=FEC/GEC

tname -- ASCII string of the team

vlan_id -- VLAN ID: from 1 to 4094, 0=untagged or no VLAN

vname -- ASCII string of the virtual device

macaddr -- MAC address (optional), e.g. 00:10:18:00:11:44

role -- Role of the physical device: 0=primary, 1=hot-standby

device -- ASCII string of the physical device, e.g. eth0

Sample Startup script for other Linux distributions:

#!/bin/bash
# load basp module
insmod basp

# create new team
baspcfg addteam 0 0 team-one

# bind physical interfaces / two primary one backup
baspcfg bind 0 0 eth0
baspcfg bind 0 0 eth1
baspcfg bind 0 1 eth2

# create the virtual interface
baspcfg addva 0 0 sw0

# bind ip address to virtual interface and initialize
ifconfig sw0 192.168.0.1 up

Startup Scripts - Redhat Distributions

  1. basp
    This script is intended to be installed in /etc/rc.d/init.d directory. After copying the script, run "chkconfig --add basp". This script is executed at runlevel 2, 3, 4, and 5. When basp runs, it searches the /etc/basp directory to list all the files with "team-" prefix, and then it invokes the "baspteam" script to add or delete the teams. It is normal, for each "team-*" file in /etc/basp to represent one team.

  2. baspteam
    This script is called by basp to add or delete a team. To install, create a "/etc/basp" directory and copy this script over.

    To manually add a team:

    baspteam team-sample add

    To delete a team:

    baspteam team-sample del

Note that "team-sample" is the configuration file.

  1. team-sample
    This script contains an SLB team configuration with three NICs: eth0, eth1 and eth2. The team name is "TeamSample". All three NICs are primary. One virtual interface is also created for this team and the name of the virtual interface is "sw0". "sw0" is the device that ifconfig should be run against to set up the IP address. No VLAN is enabled in the script.

This script and "team-GEC" are intended to be customized. See the Configuration Scripts section for details. This script should be copied to /etc/basp directory and should retain the "team-" prefix.

  1. Team-GEC
    This configuration script creates a GEC team with 3 network interfaces: eth0, eth1, and eth2. The team name is "TeamGEC". All three NICs are primary. One virtual interface is added to the team with the name "sw0" and no VLAN is enabled.

This script and the "team-sample" script are intended to be customized. See the Configuration Scripts section for details. This script should be copied to /etc/basp directory and retain the "team-" prefix.

Configuration Scripts - Redhat Distributions

Both "team-sample" and "team-GEC" are configuration scripts that follow the same syntax, as follows:

TEAM_ID: this number uniquely identifies a team
TEAM_TYPE: 0 = SLB, 1 = Generic Trunking/GEC/FEC
TEAM_NAME: ASCII name of the team
TEAM_PAx_NAME: ASCII name of the physical interface x, where x can be 0 to 7
EAM_PAx_ROLE: role of the physical interface x 0 = Primary, 1 = Hot-standby. This field must be 0 for Generic Trunking/GEC/FEC team
TEAM_VAx_NAME: ASCII name of the virtual interface x, where x can be 0 to 63
TEAM_VAX_VLAN: 802.1p VLAN ID of the virtual interface x. For untagged virtual interface (i.e., without VLAN enable), set it to 0. The valid VLAN ID can be 0 to 4094

 

Broadcom NICE Patches

Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.2.16 kernel distribution. To installed patched drivers:

  1. Copy the Broadcom NICE header file, "nicext.h", to the Linux kernel include directory,

    % cp /usr/src/nice-2.2.16/nicext.h /usr/src/linux/include/linux

  2. Rename the original network device driver under the Linux kernel source tree, "/usr/src/linux/drivers/net".

  3. Copy the patched drivers to the Linux kernel network driver source directory, for example:. "/usr/src/linux/drivers/net".

  4. Follow the kernel rebuild instructions to configure kernel support for these drivers, for example:

    % cd /usr/src/linux
    % make config

  5. If the patched drivers are configured into the kernel, goto step (7).
    If the patched drivers are configured as modules, goto step (6).

  6. In the case of supporting only the module version of these drivers, it is possible to simply run the following to compile patched drivers and to install them into the proper module directory:

    % make modules
    % make modules_install

    There is no need to compile the complete kernel. Goto step (8).

  7. Rebuild the kernel to compile these patched drivers

    % make clean
    % make dep
    % make

  8. Either reboot the system or unload/load the patched modules. Run configuration scripts to test the patch.

Uninstalling RPM Package

To uninstall the RPM package:

% rpm -e basplnx

and to reboot the system,

% reboot

Removing the Physical Interface - Generic Trunking Mode

In Generic Trunking mode, all the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface bounded to the team. In the case that this first physical interface is removed dynamically from the team using "baspcfg" tool and bounded to the protocol directly, this could lead to a duplicate MAC address problem on the network. Note that if the removed physical interface does not participate in any traffic, there will not be any problem.

To properly remove a physical interface in a Generic Trunking team, follow the steps listed below:

  1. (1) Backup the original team configuration script

    % cp /etc/basp/team-gec /etc/basp/backup-gec

NOTE 1 – "team-gec" is the name of the configuration script.
NOTE 2 – "backup-gec" is the name of the backup script. The name of the backup script must NOT be prefixed with "team-".
  1. Modify the team configuration script to remove the physical interface

  2. Stop the running team

    % /etc/basp/baspif /etc/basp/backup-gec stop
    % /etc/basp/baspteam /etc/basp/backup-gec del

  3. (4) Re-start the team

    % /etc/basp/baspteam /etc/basp/team-gec add
    % /etc/basp/baspif /etc/basp/team-gec start

Back to Top

Back to Contents


Please read all restrictions and disclaimers.