RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring

[Shyam...@dell.com]

> -----Original Message-----
> From: devel-...@open-fcoe.org [mailto:devel-...@open-fcoe.org]
> On Behalf Of Eric Multanen
> Sent: Tuesday, December 15, 2009 10:31 AM
> To: de...@open-fcoe.org
> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring
>
> v2 - restore code in the service script which destroys FCoE interfaces
> for
> which DCB is not required when the service is stopped.
>
> This patch set provides a fairly significant redesign of fcoemon.
> fcoemon
> was suffering from a number of issues, including being too reactive to
> events. This resulted in unstable behavior. This note describes the
> high
> level functions of the redesigned fcoemon. The following patches
> provide the
> implementation.
>
> fcoemon will now have the following high level structure:
> 1. Read in FCoE interface configuration files. This information is
> used
> to create a list of FCoE interfaces. As link and dcbd events
> occur and dcbd queries are resolved, the FCoE interface list will
> manage
> the creation and destruction of FCoE interfaces. Each FCoE
> interface
> element in the list keeps track of its network interface (from the
> config file and could be a VLAN), its real network interface
(found
> from link events during runtime), and its next action (create,
> destroy,
> etc.).
>
> 2. As link events are received from the system, fcoemon determines
> which ones
> are relevent for the FCoE interfaces. Relevent link events are
> those which
> occur on interfaces which are configured to be FCoE interfaces
(see
> the FCoE interface list above), or on the underlying real network
> interface,
> if the FCoE interface is configured on a VLAN. A list of relevant
> real
> network interfaces is maintained. The list is used to track the
> operstate of the real network interfaces and, if DCB is required,
> the status of dcbd queries for the real network interface.
>
> 3. The core loop of fcoemon:
> - listens for link events from rtnetlink and updates the state
of
> the
> real network interface list. Link down events will terminate
> any
> pending dcbd query sequences and a subsequent link up will
> start the
> sequence over from the beginning. Delete link events (VLAN
> deleted,
> driver unloaded) will mark the FCoE interface for destruction.
> - listens for dcbd events and response messages. If the
response
> matches the current dcbd query state, then move to the next
> state.
> If a dcbd event indicates a change in a DCB feature of
interest,
> then start the dcbd query sequence over.
> - manages a timeout for maintaining a connection to dcbd. If
the
> dcbd
> service stops, this timeout will clean up as needed and re-
> establish
> the dcbd connection once dcbd starts up again.
> - at the end of each event/timeout cycle, after all link and
dcbd
> and network interface events have been handled, perform any
> pending next actions.
> For network interface elements, that could mean sending the
> next
> dcbd query, or, if the dcbd queries are complete, analyze the
> results and determine if the FCoE interface should be
> will now handle all FCoE interfaces whether or not DCB is
> required.
> For FCoE interfaces, this means creating or destroying the
FCoE
> interface.
>
> 4. All FCoE interfaces, whether DCB is required or not, will be
> managed by
> fcoemon now. Previously, the init.d start script would handle
> configured
> FCoE interfaces which were configured without DCB required.
>
>
> dcbd query sequence
> ===================
> When DCB is required for an FCoE interface, the real network device
> performs a
> series of dcbd queries to obtain the current DCB configuration. The
> sequence
> is as follows:
> 1. DCB state - is DCB enabled on the interface?
> 2. PFC configuration (Priority Flow Control)
> 3. FCoE configuration
> 4. PFC operational state
> 5. FCoE operational state
>
> If an error occurs at any step, or if all the steps are completed and
> the
> dcbd state does not match the conditions to either create or destroy
> the
> FCoE interface (see next sections), then a retry timer is started.
> Once the
> timer expires, the dcbd query sequence is reinitiated from the
> beginning. The
> timer begins at 0.2 seconds and increases by multiples of that amount
> up to
> ten retries. Once the max retries has been reached, without a
> successful
> completion of the dcbd query sequence, then the FCoE interface is
> marked for
> destruction.
>
> The function of this retry mechanism is to protect against destroying
> the
> FCoE interface due to intermittent problems. Such as when links go
> down
> momentarily due to configuration changes. When this occurs, dcbd may
> take
> a couple seconds to resynchronize with the peer device.
>
>
> Required conditions to create an FCoE interface
> ===============================================
> 1. Link is up on the network interfaces required for the FCoE
> interface, and
>
> 2a. DCB is not required.
> OR
> 2b. DCB is required
> AND DCB is enabled
> AND PFC is enabled
> AND App:FCoE is enabled
> AND PFC and App:FCoE are operational
> AND PFC and App:FCoE operational values match
>
>
> Required conditions to destroy an FCoE interface
> ================================================
> 1. The network interface required for the FCoE interface is removed
> Such as driver is unloaded or VLAN interface is destroyed.
> This is detected by DELLINK rtnetlink events.
>
> 2. The network interfaces required for the FCoE interface are up,
> AND DCB is required,
> AND on root network interface
> DCB is disabled
> OR
> App:FCoE is disabled
> OR
> PFC and App:FCoE are operational
> AND PFC and App:FCoE operational values mismatch
>
> 3. When DCB is required and the conditions to create the FCoE
> interface are
> not satisfied after going through the maximum number of retry
> sequences,
> as described in the "dcbd query sequence" section, then the FCoE
> interface
> will be destroyed.
>
> ---
>
> Eric Multanen (11):
> Update version string of fcoemon.
> Remove FCoE interface management from service start script
> Add dcbd query timeout and retry mechanism
> Update the fcoemon state machine to be insensitive to
> intermittent dcb changes
> Update the FCoE and network interface lists.
> Add separate arguments to specify FCoE and network interface
> Fix the print_errors() routine
> Remove extraneous dcb members from the network interface
> structure.
> Fixup dcbd connection timeout code
> Remove unused dcbd routine.
> Obtain the real device of a VLAN interface using an ioctl.
>
>
> etc/initd/initd.fedora | 66 --
> etc/initd/initd.suse | 37 -
> fcoemon.c | 1504
++++++++++++++++++++++------------------
> --------
> fcoemon.h | 67 +-
> fcoeplumb.in | 121 ++--
> 5 files changed, 789 insertions(+), 1006 deletions(-)
>
> --
> Signature: Eric Multanen <eric.w....@intel.com>
> _______________________________________________
> devel mailing list
> de...@open-fcoe.org
> http://www.open-fcoe.org/mailman/listinfo/devel

Shouldn't the requirement to administer DCB be FCoE independent ? After
all DCB is required for other protocols like iscsi as well.

How about using a library interface to manage DCB over a network
interface so that both open-iscsi and open-fcoe could administer DCB?

[Mike Christie]

Shyam...@Dell.com wrote:
> Shouldn't the requirement to administer DCB be FCoE independent ? After
> all DCB is required for other protocols like iscsi as well.
>

It is not required for iscsi. I think people are thinking it is going to
be useful for iscsi though.

[Shyam...@dell.com]

> -----Original Message-----
> From: devel-...@open-fcoe.org [mailto:devel-...@open-fcoe.org]
> On Behalf Of Mike Christie
> Sent: Thursday, December 17, 2009 9:05 AM
> To: open-...@googlegroups.com
> Cc: eric.w....@intel.com; de...@open-fcoe.org
> Subject: Re: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon
> restructuring
>

Uh Ok.. I meant DCB would be useful for both FCoE and iSCSI. On the
other hand it is a requirement for FCoE.