=head1 NAME App::Netsync - network/database synchronization utility/library =begin HTML Build Status Latest Release =end HTML =head1 DESCRIPTION netsync is a network synchronization tool that: - maps network interfaces to their respective (potentially stacked) devices - gathers interface-specific information from an asset management database - sends the information it gathers to each device - provides ways of producing useful information about the network I B In a typical run, execution begins by parsing a configuration file (-c). Next, Netsync searches for active network devices at each node. Nodes may be specified by filename, DNS (-D), or the keyboard (STDIN). Furthermore, nodes may be required to match a hostname pattern (-m). Gathered information may be used to identify each device in a database. A database may be specified using Database (-d) or DB in the configuration file. Identified devices may then be updated (-u) with interface-specific information. See F for corresponding visual guidance (F after installation). =head2 0 Installation $ cpan App::Netsync or $ perl Makefile.PL $ make $ make test $ make install B =over 4 =item App::Netsync::Configurator configuration file support =item App::Netsync::Network methods for manipulating network structures =item App::Netsync::Scribe I/O framework =item App::Netsync::SNMP SNMP framework =back =head2 1 Invocation After installation, netsync may be invoked by name: $ netsync If installation is skipped, netsync may be manually invoked as follows: $ perl -I lib script/netsync B =over 4 =item -h --help Help. Print usage and options. I =item -V Version. Print build information. I =item -v Verbose. Print everything. I =item -q Quiet. Print nothing. =item -c .ini Config. Specify a configuration file. default: F =item -p # Probe. There are 2 Probe levels. If the Probe option is used, resources will be created to aid in future runs. I =over 5 =item Probe Level 1 Probe the network for active nodes. See the Probe1Cache setting. =item Probe Level 2 Probe the database for those nodes. See the Probe2Cache setting. =back =item -D DNS. Use your network's domain name system to retrieve a list of nodes. I =item -m pattern Match. Only discover nodes with hostnames matching the given pattern. =item -d .csv Database. Specify an RFC4180-compliant database file. =item -a Automatch. Enable interface auto-matching. I =item -u Update. Send interface-specific information to network nodes. =back B =over 4 =item nodes Nodes. Specify an RFC1035-compliant list of network nodes. I =back =head2 2 Configuration A configuration file may be specified using the Config (-c) option. Otherwise, a generic configuration file (F) is used, but it will most likely require additional information before running properly. Namely, the following settings must be provided for a typical installation: =over 3 =item DNS B I =over 4 =item domain network domain name (e.g. example.com) =back B See Net::DNS for more options. =item SNMP B I =over 4 =item Version =over 5 =item SNMPv3 (recommended) =over 6 =item SecLevel the level of security to deploy B =over 7 =item noAuthNoPriv (default) This causes SNMPv3 to mimic v2! =item authNoPriv =item authPriv =back =item SecName username default: initial =item AuthPass the access (authentication) key =item PrivPass the privacy (encryption) key =back =item SNMPv2 =over 6 =item Community the SNMP community to address default: public =back =back =back B See SNMP::Session for more options. =item DB B See DBI for more options. =item Netsync B Table --------------------------------------------------------- | DeviceField | InterfaceField | InfoFields... | --------------------------------------------------------- ============= | (serial) | (ifName) |(interface-specific)| ---> || SyncOID || | ... | ============= --------------------------------------------------------- (device) I B =over 4 =item DBMS the type of database (i.e. a Perl DBD, e.g. Pg, mysql, Oracle, etc.). =item Server the database network location =item Port the database transport location =item Database the name of the database to use =item Username the name of a user that has access to the database =item Password the authentication key of the user =back I =over 4 =item MIBdir the location of required MIBs default: F B =over 5 =item standard =over 6 =item IF-MIB =item ENTITY-MIB =back =item Cisco =over 6 =item CISCO-STACK-MIB =back =item Brocade =over 6 =item FOUNDRY-SN-AGENT-MIB =item FOUNDRY-SN-SWITCH-GROUP-MIB =back =item HP =over 6 =item SEMI-MIB =back =back =back =over 4 =item Table the name of the table in the database that contains the following fields =item DeviceField the field that provides a unique ID for each device (typically a serial number) =item InterfaceField the field that holds interface names retrieved from the IF-MIB (ifName) via SNMP =item InfoFields a comma-separated list of fields containing interface-specific information =back B =over 4 =item SyncOID the MIB location to store interface-specific information via SNMP =back See netsync.pl and Netsync.pm for more information. =back =head2 3 Structures Netsync builds an internal view of the network hardware whenever it is used. Each node is associated with its IP address and device(s). Each device is associated with its serial number and interface(s). Each interface is associated with its ifName and interface-specific information. The resulting data structure could be described as a list of trees. |-> node (IP) |-> node (IP) |-> node (IP) | -interface (ifName) | / | -device (serial)--interface (ifName) | / \ |-V node (IP) -interface (ifName) | \ | -device (serial)--interface (ifName) | \ | -interface (ifName) |-> node (IP) |-> node (IP) | -interface (ifName) | / |-V node (IP)--device (serial)--interface (ifName) | \ | -interface (ifName) |-> node (IP) |-> node (IP) |-> node (IP) |-> node (IP) ... B =over 3 =item inactive unreachable or unresponsive on the network =item active reachable and responsive on the network =item unrecognized active but not processable =item recognized active and processable =item unidentified recognized but not in the database =item identified recognized and in the database =back =head2 4 Discovery The first task netsync has is to find all relevant nodes on the network. Relevant nodes are specified one of three ways: =over 3 =item DNS (-D) option A zone transfer is filtered by Netsync, and a list of A or AAAA records with valid IPv4 or IPv6 addresses is produced. $ netsync -D =item Nodes parameter Nodes is a path to a file containing an RFC1035-compliant list of nodes. $ netsync -Dqp1 $ netsync /var/cache/netsync/active.txt =item STDIN (e.g. pipe or keyboard) If neither -D nor Nodes are present, a node list is pulled from standard input. This allows pipelining with other utilities for extended functionality. $ dig axfr example.com | grep hub123 | netsync Z<> $ cat superset.txt | grep hub[0-9]+ | netsync =back I $ netsync -Dm "(sw|hub)[0-9]+" Z<> $ netsync -Dqp1 $ netsync -m "(sw|hub)[0-9]+" /var/cache/netsync/active.txt Z<> $ dig axfr example.com | netsync -m "(sw|hub)[0-9]+" Z<> www.example.com <-- no match (www) hub123.example.com <-- match (hub123) sw1234.example.com <-- match (sw1234) ... B RFC1035 (3.2.1) specifies a format for resource records served by name servers. Thus, for compatibility reasons, this application is RFC1035-compliant. name ttl class type address www.example.com. 300 IN A 93.184.216.119 www.example.com. 300 IN AAAA 2606:2800:220:6d:26bf:1447:1097:aa7 ... =head2 5 Recognition netsync must attempt to contact each specified node to see if it is active. Any node that netsync attempts to contact is logged in NodeLog. If a node is active, netsync will try to retrieve its device serial number(s). If multiple devices are discovered, netsync will try to map each interface to its device (serial). If no devices are discovered (or mapping is impossible), The node is considered unrecognized and skipped. I B =over 3 =item Brocade (stacks) =item Cisco (stacks) =item HP =back B =over 3 =item 1 Extract interfaces. B =over 4 =item 1 ifType (1.3.6.1.2.1.2.2.1.3) B =over 5 =item 1 other =item 24 softwareLoopback =item 53 propVirtual =back =item 2 ifDescr (1.3.6.1.2.1.2.2.1.2) I =back =item 2 Extract serials. B =over 4 =item 1 entPhysicalClass (1.3.6.1.2.1.47.1.1.1.1.5) B =over 5 =item 3 chassis =back =item 2 entPhysicalSerialNum (1.3.6.1.2.1.47.1.1.1.1.11) =back B =over 4 =item Brocade snChasUnitSerNum : 1.3.6.1.4.1.1991.1.1.1.4.1.1.2 I =item Cisco moduleSerialNumber : 1.3.6.1.4.1.9.5.1.3.1.1.3 I =item HP hpHttpMgSerialNumber : 1.3.6.1.4.1.11.2.36.1.1.2.9 =back =item 3 Map interfaces to serials. B I B =over 4 =item Cisco =over 5 =item 1 portIfIndex 1.3.6.1.4.1.9.5.1.4.1.1.11 =item 2 portModuleIndex 1.3.6.1.4.1.9.5.1.4.1.1.1 =item 3 moduleSerialNumber 1.3.6.1.4.1.9.5.1.3.1.1.3 I =back =item Brocade =over 5 =item 1 snSwPortIfIndex 1.3.6.1.4.1.1991.1.1.3.3.1.1.38 =item 2 snSwPortDescr 1.3.6.1.4.1.1991.1.1.3.3.1.1.39 I =item 3 snChasUnitSerNum 1.3.6.1.4.1.1991.1.1.1.4.1.1.2 =back =back =back =head2 6 Probe Level 1 Probe Level 1 is specified using -p1 and updates Probe1Cache. During Probe Level 1, only the discovery stage is executed. It caches discovered nodes in Probe1Cache. This cache is created in RFC1035-complaint format. It may subsequently be passed as the Nodes parameter to skip inactive nodes. $ netsync -Dm "sw[^.]+|hub[0-9]+" -p1 > configuring (using /etc/netsync/netsync.ini)... > discovering (using DBMS)... 480 nodes (10 skipped), 500 devices $ netsync /var/cache/netsync/active.txt > configuring (using /etc/netsync/netsync.ini)... > discovering (using /var/cache/netsync/active.txt)... 480 nodes, 500 devices > identifying (using DBMS)... 498 synchronized (4 conflicts) I =head2 7 Identification Once netsync has a view of the network's hardware, it requires a database to find information specific to each interface. I The database may be provided one of two ways: =over 3 =item DBMS This must be preconfigured in the configuration file and on the DBMS. =item Database (-d) An RFC4180-compliant database file may be specified. =back netsync identifies the network interface referenced in each database entry. A valid entry must not have missing DeviceField or InterfaceField information. Valid rows are synchronized with the network while invalid rows are skipped. Entries with unknown (not found) devices are skipped. Entries are then checked for conflicts. B RFC4180 specifies a simple format (CSV) for database files. This format is widely supported making it useful for importing and exporting. Thus, for compatibility, this application is RFC4180-compliant. DeviceField,InterfaceField,InfoField A1B2C3D4E5F6,ethernet1/1/1,PoE Enabled A1B2C3D4E5F6,ethernet1/2/1,Uplink ... =head2 8 Synchronization All conflicts are logged in ConflictLog and skipped. B =over 3 =item Mismatch This occurs when an InterfaceField value can't be found on an identified device. =item Duplicate This occurs when more than 1 entry for an interface exists in the database. =item Unidentified This occurs when hardware is found on the network but not in the database. =back =head2 9 Probe Level 2 Probe level 2 is specified using -p2 and updates Probe1Cache and Probe2Cache. During Probe Level 2, only the discovery and identification stages are executed. It caches identified nodes in Probe2Cache. This cache is created in RFC4180-complaint format. It may be passed as Nodes in subsequent runs to skip synchronization. $ netsync -Dm "host[0-9]+" -a -p2 > configuring (using /etc/netsync/netsync.ini)... done > discovering (using DNS)... 480 nodes (10 skipped), 500 devices > identifying (using DBMS)... 498 synchronized (4 conflicts) > Do you want to resolve conflicts now? [y/n] n $ netsync -d /var/cache/netsync/synced.csv /var/cache/netsync/active.txt > configuring (using /etc/netsync/netsync.ini)... done > discovering (using /var/cache/netsync/active.txt)... 480 nodes, 500 devices > identifying (using /var/cache/netsync/synced.csv)... 500 synchronized I =head2 10 Updating All modifications made to any device are logged in UpdateLog. If probing is not used and the Update (-u) option is specified, netsync attempts to actualize its internally synchronized network using SNMP. It pushes gathered interface-specific information to the devices on the network. This information is stored in the device at the specified SyncOID, and is overwritten anytime netsync updates it. $ netsync -Dau =head1 EXAMPLES $ netsync -Dm "sw[^.]+|hub[0-9]+" -au > configuring (using /etc/netsync/netsync.ini)... > discovering (using DNS)... 780 nodes (50 skipped), 800 devices (10 stacks) > identifying (using DBMS)... 670 synchronized (4 conflicts) > updating... 670 successful Z<> $ dig axfr domain.tld | egrep ^(sw[^.]+|hub[0-9]+) | netsync -a > configuring (using /etc/netsync/netsync.ini)... > discovering (using STDIN)... 780 nodes (50 skipped), 800 devices > identifying (using DBMS)... 670 synchronized (4 conflicts) Z<> $ netsync -Dm "sw[^.]+|hub[0-9]+" -p1 > configuring (using /etc/netsync/netsync.ini)... > discovering (using DBMS)... 780 nodes (50 inactive), 800 devices $ netsync /var/cache/netsync/active.txt > configuring (using /etc/netsync/netsync.ini)... > discovering (using /var/cache/netsync/active.txt)... 780 nodes, 800 devices > identifying (using DBMS)... 670 synchronized (4 conflicts) Z<> $ netsync -Dm "sw[^.]+|hub[0-9]+" -a -p2 > configuring (using /etc/netsync/netsync.ini)... > discovering (using DNS)... 780 nodes (50 skipped), 800 devices > identifying (using DBMS)... 670 synchronized (4 conflicts) $ netsync -d /var/cache/netsync/synced.csv /var/cache/netsync/active.txt > configuring (using /etc/netsync/netsync.ini)... > discovering (using /var/cache/netsync/active.txt)... 780 nodes, 800 devices > identifying (using /var/cache/netsync/synced.csv)... 800 synchronized =head1 AUTHOR David Tucker, C<< >> =head1 BUGS Please report any bugs or feature requests to C, or through the web interface at L. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. =head1 SUPPORT You can find documentation for this module with the perldoc command. perldoc App::Netsync You can also look for information at: =over 4 =item * RT: CPAN's request tracker (report bugs here) L =item * AnnoCPAN: Annotated CPAN documentation L =item * CPAN Ratings L =item * Search CPAN L =back =head1 LICENSE Copyright 2013 David Tucker. This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License. See L for more information. =cut