--- /root/netmagis/expl/generer-zone 2011-04-29 13:25:30.000000000 +0200
+++ /usr/local/sbin/generer-zone 2011-04-29 13:32:16.000000000 +0200
@@ -649,8 +649,8 @@
return 1
}

• set sql "BEGIN WORK ; LOCK zone ; \

•           LOCK zone_normale ; LOCK zone_reverse4 ; LOCK zone_reverse6"
  

• set sql "BEGIN WORK ; LOCK dns.zone ; \

•           LOCK dns.zone_normale ; LOCK dns.zone_reverse4 ; LOCK dns.zone_reverse6"
  

  if {! [execsql $dbfd $sql m]} then { puts stderr $m ; exit 1 }

  switch [llength $argv] {

The paper format should be a configuration parameter.

Modify ./metro/ subtree to new file hierarchy specified in HIER file,
create a ./metro/Makefile with three targets (all, clean, install),
and modify ./common/etc/netmagis.conf to include metro keys.

At this time, SOA number has a fixed format : yyyymmddnn, which allows only 100 modifications a day, which constrains the crontab to not run the mkzone script too often.
The idea is to make the SOA a pure sequential number, which allows for a much more tight delay.

• extension of the "eq" table in order to add attributs "model" and "comment" :

^ idrr ^ type ^ up/down ^ model ^ comment ^

• attribute "model" contains model revision of the equipment (e.g. Cisco 295024T)
• attribute "model" is automatically feeded by TOPO module
• attribute "comment" is for free comments (e.g. Nagios templates)
• attribute "comment" could feed a supervision or inventory system.
• creation of the "nagios.templ" table :

^ type ^ model ^ templatename ^

• nagios.templ.model refers to the eq.model
• template stores the hardware specific template for Nagios/Centreon
• we could develop a new module called "Nagios" in order to build interfaces for exporting data from Netmagis. This module strores scripts for massive change and modification of nagios.templ.templatename attribute and eq.comment attribute.

Replace metro.sensor file using topo.sensor table
(or an union of topo.sensor and a new table metro.sensor
for manually created sensors).

Remove metro.graph file using topo.sensor table too.

User should be able to see all stats for a given equipment together on one unique page.

Formalize (and automatize) the release engineering process:

• document beta cycles
• check-list of file modifications (NMVERSION in Makefile, CHANGES, README, etc.)
• automate source tgz generation (done already for v2.0)
• automate FreeBSD and Debian port/package/packet generation
• automate documentation installation
• document announcement (mailing lists, web sites, etc.)
  [list to be completed]

Some non-admin users would like to write their own DHCP profile.
DHCP profile should be validated before entering the database, since an invalid DHCP profile may prevent the ISC DHCP server to restart. For this validation, an ISC DHCP server must be installed on the WWW server, and the DHCP configuration file must be available (for example stored in the database).

Integrate MAC module in netmagis.

Toposendd daemon sends configuration commands to equipments. These commands are specified in an array in the toposendd program.
These commands should instead be located in a database table in order to let users change them easily.

Do this periodic cleanup in the tables in toposendd and topographd daemon.

Topo informations:

• are refreshed with a delay
• are structural informations only (links, vlans, etc.).

There is a need to display more dynamic informations, such as:

• interface speed (real speed)
• adminstatus (shutdown or alive)
• operstatus (connected or not)
• MAC addresses on this port

The installation documentation must be completed with:

• metro and mac installation
• how things work
• common tasks

Fix broken Wifi graph aggregation

Provide packet files for Debian/Ubuntu
Rework installation documentation accordingly

pdflatex is used by the Web application to generate printable documents.
Path to pdflatex is hard-coded in common/pkgtcl/webapp.tcl.

Virtual equipments are an essential part of the topo module in order to connect components of the graph through not maintained or not supported equipments such as various Linux/FreeBSD firewall or unknown equipments.
At this time:

• virtual equipments are not documented anywhere
• they must be provided by the user

A reasonnable target for v2.1 is to document at least how to write virtual equipments.

A ambitious target is to model these equipments in the database via an easy-to-use GUI.

The idea is that rights are quicker to apply at the cgi level
on an equipement view (produced by "extracteq -a") :
there are only a few interfaces to test (worst case is a few dozens
interfaces). This is how it can be done :

• create a new table, r_vlan (vlan rights) with 3 columns : auto (0/1), group id, vlan id
• some vlan rights can be added manually (auto=0)
• update the table after each graph build : replace all lines for a given group (where auto=1) with all the vlans extracted from the graph using the authorized networks. This is done without applying any equipment pattern rights.
• modify function eq-iflist in libdns to apply read and write rights with TCL code instead of pulling them from extracteq
• other benefit : this allows for manually added vlan rights. Current vlan rights are derived from network rights, which require a routing point in each vlan.
• drawback : right modification are not immediatly seen ; a graph build must occur to refresh vlan rights

Toposendd configuration templates (commands to send to equipements in order to modify interface characteristics) are located in an array in the program code.
This should be transfered to database, to ease template editing.

The "eq" page is rather when an equipment has many vlans on tagged interfaces.
An useful feature would be to collapse by default all vlans if number of tagged vlans on a given interface is above a threshold.

Toposendd daemon never re-read equipment types.
Two ideas to do so:

1. topographd could notify (via a fake entry in topo.ifchange) when graph is rebuilt, or
2. toposendd could monitor graph file

Speed of topo programs should be improved.

Idea: group all that is needed by a single Topo CGI script in only one program. This could save multiple graph traversal to select allowed interfaces/equipement.

1. Keep rancid local patches in git repository
2. Contact rancid author to include patches in mainstream

These icons are referenced by the "eq" CGI script, via the ::webapp::interactive-tree function of ./common/pkgtcl/webapp.tcl file.
Returned link specifies the path "images", which should be "css/images".

This issue should be investigated.

Topo boot scripts launch toposendd and topographd daemons.
At this time, they are pretty generic.
Specialized versions must be designed for FreeBSD and Debian/Ubuntu systems.

The need is to improve DNS zone generation latency.

Some ideas to achieve this goal:

1. add a configuration item to parametrize the DNS zone generation frequency and display a consistent message when a RR modification is done through the Web interface. In this case, the cron frequency (call to mkzone) will continue to be set manually. Consistency between the new parametrized value and the cron value will continue to be assured manually.
2. write a dns update daemon, instead of scripts run by cron. The new daemon could use an adaptative frequency: given two digits for the version number, this allows for a modification every ~14 minutes. While modification rate is lower than this, use a small delay. When modification rate is above this figure, lower the delay.
3. use a different version number. For example: 3000 and increment nnnnnn (without reset it each day)

Integrate Metro into WebDNS

• integrate Metro repository
• write a installation documentation
• write a poller to get forwarding table (for an equipment and a vlan) in order to give a couple (mac, interface)
• write a poller to get ARP/ND tables (for an equipment) in order to give a couple (mac, IP address)
• remove all Osiris specific items from metro
• use the WebDNS database. For example, sensors should be stored entirely in the topo.sensors table.

Non priviledged users should be authorized to see contents of their DHCP profiles.

Create new sensor types:

• B... : broadcast + multicast
• E... : traffic errors

Il est proposé une interface Web partant de l'actuel topo (/bin/eq) permettant à partir de la page d'un équipement, de sélectionner une ou plusieurs interfaces pour modifier :

• sa description
• le vlan d'accès
• le vlan VoIP

Cette application permettra de configurer une interface sous les conditions suivantes :

• l'interface sert à connecter un équipement terminal (PC et/ou téléphone) et non un équipement réseau. Il n'est donc pas question de modifier les liens d'interconnexion du réseau.
• le vlan demandé doit être présent sur l'équipement [pour mémoire, il vaut mieux que tous les vlans aient été provisionnés sur les équipements, suivant les décisions prises lors de la réunion du 14/09/2010]
• l'application sera ouverte aux correspondants réseau
• les droits d'accès sur les vlans sont déduits des droits d'accès sur les sous-réseaux IP associés (modèle de droits de topo dérivé de WebDNS)

Mécanisme :

• les modifications sont décrites dans la base de données
• un démon vérifie périodiquement (< 5 min) les demandes de modification, les envoie vers les équipements, et reconstruit le graphe de la topologie
• la reconstruction du graphe est une opération lente
• l'affichage des interfaces d'un équipement sera complété avec les modifications demandées mais non encore propagées sur l'équipement
• conclusion : les demandes ne sont pas immédiatement répercutées vers les équipements, une petite latence est à prévoir.

Note : ce point de modification des interfaces des équipements nécessite une refonte particulièrement importante de l'infrastructure de la topologie.

Topo information such as network equipment and port should be added to the list of hosts in a network.

In hierarchical view of equipments, if equipment "a-b1" is given after "a-b", then the CGI script "eq" fails.

From a fresh installation, topographd stops with the following error

no files matched glob pattern "/local/var/netmagis/eqvirt/.eq"
while executing
"glob "$dir/.eq""
(procedure "detect-dirmod" line 10)
invoked from within
"detect-dirmod $eqvirtdir err"
(procedure "main" line 119)
invoked from within
"main $argv0 $argv"
invoked from within
"exit [main $argv0 $argv]"
(file "/usr/local/lib/netmagis/topo/topographd" line 858)

The netmagis.conf file should contain a single line:
rancidbasedir
instead of ranciddb + rancidconfdir.

All the commands sends to the network equipements are stored in an array in the
script "topo/src/toposendd". We need an easy way to modify or extend the commands.
So this should be independant from the codebase.

The commands are just data ; as such, they should be put into a separate configuration file.

CSV export is already available.
CSV import is more complex since it needs some rules to garantee consistency without adding too much constraints for the user. Examples of rule: don't allow a second address to an existing host, alias management, etc.

Provide manual pages for all scripts installed in .../bin and .../sbin

Provide and update a CHANGES file to keep track of changes in Netmagis versions.

Cette option est accessible à partir de plusieurs pages :

• page d'ajout de machine (/bin/ajout), section "Recherche de plusieurs adresses IPv4 disponibles"
  • Option [Rechercher] : permet de sélectionner une plage de N adresses consécutives et d'effectuer l'ajout des N machines
  • Option [Consulter la carte] : affiche une carte du sous-réseau colorisée suivant le statut des adresses IP
• page de consultation des réseaux (/bin/consulter)
  • Option [Carte des adresses] : affiche la carte du sous-réseau

À partir de la carte du sous-réseau :

• cliquer sur une adresse non déclarée permet de déclarer une machine
• cliquer sur une adresse déclarée permet d'éditer les informations (type de machine, adresse MAC, etc.) ou de supprimer la machine

Statut :

• codage effectué, installé dans l'appli de test (cf url de type devdns)
• en attente de validation/information pour le déploiement

There are numerous crutches in the code to convert back and forth FQDN to short names.
We should use FQDN everywhere (except on display, may be).

Check and analyze performance of the topo module:

• graph building (anaconf program)
• graph use (eq/l2/l3 CGI scripts)

All C programs called during these scripts must be analyzed with gprof or similar on a very large network to see if they scale.

The idea is to match physical location of network equipements with any reference database of buildings,
by using the snmp sysLocation variable or similar.

Since they are a single program, provide only one netmagis.conf directive (e.g. the dot program).
Graph drawing engine should be supplied by a parameter given to dot such as "dot -Glayout=neato ..."

Adapt FreeBSD ports to new source configuration resulting from Debian packaging.
Adapt installation documentation if needed

It would be great if it was possible to create several views for a given zone into Netmagis. Once a view is created, it would be possible to add a host into it and to generate a zone file for this view.

Then, it would be easy (by configuring bind) to serve a different dns view depending on your location on the network.

Some parameters names are not displayed correctly by the www/cgi/admpar script (i.e. key names not translated) both in English and in French.

Various parts of netmagis produce some logs.
Log production should be harmonized.
Log exploitation/display should be centralized at one point in the Web application for admin users.

Rework Web admin scripts for ergonomy.
The new admin scripts should be "task oriented" :

• I want to add a new network or zone (new network/zone => new access rights)
• I want to remove a network or zone (delete / rename old hosts, remove access rights, etc.)
• I want to renumber a network (change IP addresses of all hosts, change access rights, etc.)
• etc.

Existing admin scripts should be kept for "raw" administration.

Cette option est accessible depuis le bandeau de gauche de WebDNS (lien "Où suis-je ?")

Affichage de l'adresse du client Web

• si hors des réseaux gérés : affichage de l'adresse IP
• si dans un réseau géré, et adresse non déclarée : affichage des correspondants responsables du sous-réseau
• si dans un réseau géré, et adresse déclarée : affichage du nom et des correspondants responsables du sous-réseau