nictool / nictool Goto Github PK

1. the DB schema and web interface already have start timestamps (a tinydns-only feature)
2. add an expire/until timestamp, after which records stop getting published
3. update export routines to:
   • check for timestamps within the last_export <=> now() range
   • if any expires or creates, trigger new export
4. test with each set of exports

This change was made in RFC 4035.

The "PBKFD2" probably should read "PBKDF2" and might be worth fixing at least in the release notes.

When a common setting (like 5 minutes, or 1 hour, or 1 day) is chosen, update the input field with the corresponding number of seconds.

Problem: A somewhat rare problem. The way to demonstrate it as follows:
Load up a NicTool install with 500,000 zones on an old slow server. Time how long an export takes to run. Set the Export Interval for that NS to lower than the time it takes to export.

Solution: store the PID of the nt_export process in the NS table when exporting. Remove it when completed. When starting to export, check the database. If a PID exists, see if that process is currently running. If not, update and continue. If so, sleep for a minute.

Make sure they work correctly.

RFC 3490, IDNA (International Domain Names in Applications)
RFC 5890 - 5894 (IDNA 2008)

May 14 13:30:53 [PIPEBackend] coprocess returned incomplete MX/SRV line in data section for query for 3658
May 14 13:30:53 TCP nameserver had error, cycling backend: Format error communicating with coprocess in data section of MX/SRV record

This is with PowerDNS 3.3.1 from the source tarball and Nictool 2.21 when doing an AXFR. All other queries return NXDOMAIN .

Hi, when I run zone2nic.pl, I get
(nictool ) 0 # ./zone2nic.pl -z one.zone -s our.nameserver -a
Name "NicToolServerAPI::use_https_authentication" used only once: possible typo at ./zone2nic.pl line 62.
Logged in as test test
Importing one.zone:
Cannot encode unnamed element as 'hash'. Will be encoded as 'map' instead
300 - Sanity error: The mailaddr format replaces the @ with a . (dot).
*** Failed to create one.zone: Sanity error : The mailaddr format replaces the @ with a . (dot).

But I don't know where the problem is.
The zone doesn't contain any "@" in the SOA record or in any other record.

I've commented out the check and now I can at least import stuff via zone2nic.pl.
Our zones should be OK in that respect.

This is with PERL 5.18 (on my "dev" install locally in Fusion), but I get the same error at work on my slightly older PERL 5.16 install (all FreeBSD 10).

Fix bug: login returns nt_perm_id and perm_name in user hash, and should not.

For efficiency, when exporting to BIND, only export zones that have changed since the last successful export.

A soap call made to that method results in a new entry being created even though a record id was supplied.

This then results in a series of duplicated entries.

Add support for other database engines. This requires:

• removing all MySQL specific ENUM columns (mostly done as of v2.10)
• Support NoSQL dbs like Redis and MongoDB

I haven't figured out whats going on yet but it looks to be broken.

[root@dns server]# ./bin/nt_import.pl -file ./bin/data -user shaun.reitan -verbose -type tinydns
Please enter nictool pass: loading type: tinydns
file: ./bin/data
SOA: mydomain.com:ns1.mydomain.com.:hostmaster.ZONE.TLD.:2007072904:16384:2048:1048576:2560:86400::
creating zone mydomain.com
Sanity error ( missing label AND A domain name must have at least 1 octets (character): RFC 2181 ), $VAR1 = {
          'contact' => 'hostmaster.ZONE.TLD.',
          'description' => '',
          'expire' => '1048576',
          'minimum' => '2560',
          'refresh' => '16384',
          'retry' => '2048',
          'ttl' => '86400',
          'zone' => 'mydomain.com'
        };
NS : mydomain.com::ns1.mydomain.com.:86400::
NS : mydomain.com::ns2.mydomain.com.:86400::
MX : mydomain.com.::mail.mydomain.com.:10:3600::
could not find zone for mydomain.com

here's my data file

Zmydomain.com:ns1.mydomain.com.:hostmaster.ZONE.TLD.:2007072904:16384:2048:1048576:2560:86400::
&mydomain.com::ns1.mydomain.com.:86400::
&mydomain.com::ns2.mydomain.com.:86400::
@mydomain.com.::mail.mydomain.com.:10:3600::
Cwww.mydomain.com:mydomain.com.:3600::
+mail.mydomain.com:204.10.37.96:3600::
+mydomain.com.:204.10.37.96:3600::

As of 2.10, NicTool has partial location (split-horizon) support. That means you can publish DNS records that are answered based on the askers IP. But...

1. the web gui hasn't been updated to allow editing location definitions
2. there is no location to store the location definitions
3. which means tagging records with locations is a manual SQL process
4. location definitions must be manually added to the exported data file
5. I'd do that with some entries in the Makefile

Before I go any further, I'd like some feedback/ideas on how you would use it, and suggested implementations and ideas.

With tinydns, location definitions affect the entire nameserver. With BIND views, they can be specified on a zone-by-zone basis. Ideally NicTool will provide a mechanism that works for either. One idea is to have a single set of location definitions. Zone records would get a drop down list of locations they could choose from (world-vs-private). Implementing this for tinydns exports would be cake. It's not too bad for BIND either.

Before location support can be exposed via the API and web interface (NicToolClient), the Zone and Record sanity checks need to be updated. Many of the record restrictions need to take location into consideration.

RFC 2181, RRSet restrictions
If RR has identical label, type, and data as existing RR, reject as invalid

If a user does not have delete permission for a zone/zone record, they should not have restore permissions

See https://www.tnpi.net/support/forums/index.php/topic,797.0.html

I have something related that I'd like you to consider. Most NicTool installs run the export processes for all of their name servers on the NicTool host. Example:

1. nictool.example.com
2. ns1.example.com
3. ns2.example.com
4. ns3.example.com

Quite often, the NicTool server also serves as one of the name servers. So, in this case, nictool is actually a CNAME of ns1. When I do this, things are going to work out poorly:

cd /usr/local/nictool/ns2.example.com
perl ./nt_export.pl

I won't get a list of NSIDs to choose from, and I won't easily be able to get at them either.

I have just recently installed nictool 2.22 and found when i attempt to edit a user via the nictool client i was receiving an internal server error with the following:
[:error] [pid 23956] Global symbol "$data" requires explicit package name at /var/www/html/nictool/user.cgi line 497.\n

A quick look at the code and i found $data being used where it looks like $duser should be, i have fixed it on my install and it seems functional now....

http://tools.ietf.org/html/rfc3597

Add export support for unknown DNS record types.

Hi!

I have hit a following problem on my testing instalation of NicTool: I tried to set a password for user to fairly complex randomly generated string. For example edSuWHwjZXcKlts6 . It worked well, no error during password setting. But I was unable to login with this password afterwards.

Is there some maximum password length?

Thanks,
Tomas

E.g. for names by type to do a mass update of the ttl. (Jörg Site)

I have just installed Nictool V. 2.31 and import is not longer working. I can confirm the same issues with the git version.

root@amnesio:~/home/namedb# perl /usr/local/nictool/server/bin/nt_import.pl --user=root --pass=XXXX  --type=bind --file /home/pto/home/namedb/gen/named.pto-test.conf --verbose
loading type: bind
$VAR1 = {
          'deleted' => '0',
          'email' => '[email protected]',
          'error_code' => '200',
          'error_desc' => '',
          'error_msg' => 'OK',
          'first_name' => 'Root',
          'group_create' => '1',
          'group_delete' => '1',
          'group_write' => '1',
          'groupname' => 'NicTool',
          'inherit_group_permissions' => '1',
          'inherit_perm' => undef,
          'is_admin' => undef,
          'last_name' => 'User',
          'nameserver_create' => '1',
          'nameserver_delete' => '1',
          'nameserver_write' => '1',
          'nt_group_id' => '1',
          'nt_user_id' => '1',
          'nt_user_session' => '54d8c939187d044c',
          'pass_salt' => 'uDN*Ql6DKP](KdSO',
          'self_write' => '1',
          'usable_ns' => '2',
          'user_create' => '1',
          'user_delete' => '1',
          'user_write' => '1',
          'username' => 'root',
          'zone_create' => '1',
          'zone_delegate' => '1',
          'zone_delete' => '1',
          'zone_write' => '1',
          'zonerecord_create' => '1',
          'zonerecord_delegate' => '1',
          'zonerecord_delete' => '1',
          'zonerecord_write' => '1'
        };

Starting import using: /home/pto/home/namedb/gen/named.pto-test.conf
zone: ip.cybercity.dk   from    gen/11/P.ip.cybercity.dk
creating zone ip.cybercity.dk
Can't locate object method "nameservers" via package "NicToolServer::Import::BIND::Conf_Parser" at /usr/local/share/perl/5.14.2/NicToolServer/Import/Base.pm line 92.
root@amnesio:~/home/namedb#

add an option to update PTR when editing A or AAAA records

(if reverse zone exists and user has permission to edit PTR)

RFC 2181, RRSet restrictions
.. Make sure TTL on RRs with identical label and type are identical

SHA-1 use is deprecated and should be replaced with bcrypt or scrypt, which is better suited for encrypting passwords. See http://security.blogoverflow.com/2013/09/about-secure-password-hashing/, http://en.wikipedia.org/wiki/Scrypt, http://en.wikipedia.org/wiki/Bcrypt

I had everything working earlier today, decided to blow it away and start again with the most recent commits in place. Now I'm getting internal server errors from Apache upon initial login with the root user.

to reproduce:

1. install NicTool client and server;
2. mysql> drop database nictool;
3. nictool# /usr/local/nictool/server/sql/create_tables.pl # after adding the appropriate values; exits successfully with no errors.
4. log in to the nictool web UI with the root username you just provisioned and get an internal server error from apache, with the following text in the error_log:

[Tue Nov 12 22:18:12 2013] [notice] Apache/2.2.15 (Unix) DAV/2 mod_ssl/2.2.15 OpenSSL/1.0.0-fips mod_perl/2.0.4 Perl/v5.10.1 configured -- resuming normal operations
Client error: 302: nt_group_id: Some parameters were invalid ModPerl::ROOT::ModPerl::Registry::usr_local_nictool_client_htdocs_nav_2ecgi:/usr/local/nictool/client/htdocs/nav.cgi:48 at /usr/local/nictool/client/lib/NicToolClient.pm line 1316.
[Tue Nov 12 22:18:21 2013] [error] Can't use an undefined value as an ARRAY reference at /usr/local/nictool/client/lib/NicToolClient.pm line 245.\n

the content of the error message sounds very similar to the SOAP bug referenced on the nictool server install doc, but I've already made that fix and confirmed it works. I can log in, but it really looks like there's something that should be defined in the database by create_tables.pl that isn't being set properly. (among other things, I found a few more places where nt_group_id=1 was referenced when it should have been nt_group_id=0, per the recent patch to that effect.)

[Tue May 20 13:37:51 2014] [error] Global symbol "$data" requires explicit package name at /var/www/nictool/client/htdocs/user.cgi line 497.\n
[Tue May 20 13:37:55 2014] [error] Global symbol "$data" requires explicit package name at /var/www/nictool/client/htdocs/user.cgi line 497.\n
[Tue May 20 13:38:57 2014] [error] Global symbol "$data" requires explicit package name at /var/www/nictool/client/htdocs/user.cgi line 497.\n

When clicking on a user .

vtsingaras at it dot auth dot gr

With the demise of dyndns.org as a useful free service, add a script to the nictool package that a client (freebsd, mac, linux) can run to keep a hostname updated with their public IP.

This looks to be happening all over the code and i'm guessing this is a bug, either that or i'm using these functions wrong. I discovered this first by using NicTool.pm and calling $r = $nt->get_nameserver( nt_nameserver_id => X); on a nameserver id that does not exist. I expected to be able to use $nt->is_error($r) to check for a error but instead i found myself getting a Use of uninitialized value in concatenation (.) or string at and when using data::dumper on $r i see the following...

                 'store' => {
                              'error_msg' => 'Can\'t use an undefined value as a HASH reference at /usr/local/share/perl5/NicToolServer/Nameserver.pm line 251.
',
                              'error_desc' => 'Internal Error',
                              'error_code' => '508'
                            },

So a couple things look to be happening, one being that an error is happening but for some reason is_error is not catching it, and 2 being that the module is erroring because it expected a return. I'm thinking this could be easily fixed by adding a check to the get_nameserver function that checks if any results were returned from the database. I can try and fix these as i find them on my own but i didnt want to put in a bunch of work if I was wrong. Also, what error could should be sent if a 'not found' error happens? Or maybe no error is returned and a empty result is returned? Let me know your thoughts.

In your upgrade script you have a section that reads...

 /* doesn't matter if this fails, b/c it was already present */
ALTER TABLE nt_user ADD COLUMN is_admin TINYINT(1) UNSIGNED default '0' AFTER email;

The problem is that it does infact matter if it fails because the upgrade script fails giving the admin the false idea that the upgrade script finished when it has not. Example and error below.

 /* doesn't matter if this fails, b/c it was already present */
ALTER TABLE nt_user ADD COLUMN is_admin TINYINT(1) UNSIGNED default '0' AFTER email;
DBI error: Duplicate column name 'is_admin' at ./upgrade.pl line 61, <STDIN> line 2.
[root@dns sql]#

Idea proposed by Arthur G:
"In the NicTool Client's Name Server configuration screen just add a "Check NS is serving latest data" button (that's a big button), which calls a Perl class that runs your SQL query and performs a DNS lookup of the serial using some Perl API. If up-to-date, display a green tick, otherwise display a red cross with the two zones with their serials. Also cater for a timed-out result for people that leave old name servers inside the Nictool configuration or misconfigure their networks."

I started an export this morning and left it running while my family and I went off to a corn maze for a few hours. When I got back, my SSH session has been dropped and I hadn't backgrounded the process so the export was terminated. When I started the export process back up, it started again from the beginning.

NicTool's exports have historically revolved around per-NS exports. Either we're able to publish all our changes to that particular NS, or we aren't. If not, try again. That model has worked quite well for 15 years, particularly as zones have a one-to-many relationship with nameservers. NS exports typically complete in seconds or minutes. For Dyn exports, that model doesn't work.

An export with thousands of zones will take hours or days, partially because of API request throttling. Even ignoring this "one time export" event of extraordinary duration, the DynECT exports are no longer NS specific and instead publish to Dyn once to update a group of nameservers. Further, an export run is no longer a binary transaction in which every export succeeds or fails. Now an export run can consist of zones that published, zones with errors, and zones that fail. Such a large export will almost never succeed, causing subsequent attempts to republish all zones.

Therefore, in the not-too-distant future, a new version of NicTool will be released with a simple change. The nt_zone table will get a new publish_ts field. Immediately after each zone is successfully published to Dyn, that zones publish timestamp will be updated. Export processes use a SELECT ... WHERE z.last_modified > z.last_publish query to select only zones that have changed since their last successful export.

For the various DNSSEC record types, provide popup menus, populated with the human friendly labels that set the corresponding DNS values.

Support added in v2.18

For managing a cluster of Bind servers, without the need to restart the service:

• Create and delete records: nsupdate
• Create and delete subzones: rndc

We have an user that deleted a whole zone, and then recreated it as new... which led to two entries in nt_zone for the same zone name: one deleted, one active, which is correct.

Unfortunately, when running the export script (for our case BIND), an strace shows me that the zone file gets created, and then later it gets deleted. :)

I suggest that the cleanup loop in export_db() (in Base.pm) should be done before the export of the zones.

Thanks

SPF records: when creating or editing a SPF record, check for an TXT record.
If TXT doesn't exist, create it. If exists, update it with SPF record.

RFC 4408 ("recommends to always provide a TXT based SPF RR...and duplicate the information")

Two things with SRV records.

• First, it has a missing validation for the port number. It accepts it when black, which leads to a broken BIND zone file.
• Second, whenever we want to edit an existing SRV record, the "weight", "priority" and "port" fields are not being filled with the pre-existing values.

Thanks

Currently, during DNS imports, NS records are ignored. These are handled by a command line argument nt_import.pl --nameservers=[nsid, nsid, nsid]. This has the unfortunate side effect of ignoring NS records inside a zone file that aren't for that zone (glue records). A band-aid would be to check the hostname of the record and only ignore the NS if the fully qualified hostname equals the zone name. A better way to handle this would be:

1. allow zones to be created with zero name servers Done.
2. fetch a list of name servers when starting the import
3. when NS records are encountered, and the name records is the same as the zone name, check the NS name against the list. For NS records in the list, insert the appropriate entry into nt_zone_nameserver. For all else, create the NS record as for any other RR. Done.
4. remove the --nameservers option from nt_import.pl Done.

However the edit_zone_record API call does return nt_zone_record_id

Not sure about the other edits for nameserver and user...

If i get a sec i'll look into the and put up a diff patch.

Nictool version: 2.30
Operation system: Ubuntu 12.04 LTS
Perl: v5.14.2
nt_install_deps.pl: all modules installed

cd /home/pto/home/namedb
perl /usr/local/nictool/server/bin/nt_import.pl --verbose 1 --nameserver=4 --user=root --pass=XXXXX --type=bind --file /home/pto/home/namedb/gen/named.pto-test.conf

loading type: bind
$VAR1 = {
'deleted' => '0',
'email' => '[email protected]',
'error_code' => '200',
'error_desc' => '',
'error_msg' => 'OK',
'first_name' => 'Root',
'group_create' => '1',
'group_delete' => '1',
'group_write' => '1',
'groupname' => 'NicTool',
'inherit_group_permissions' => '1',
'inherit_perm' => undef,
'is_admin' => undef,
'last_name' => 'User',
'nameserver_create' => '1',
'nameserver_delete' => '1',
'nameserver_write' => '1',
'nt_group_id' => '1',
'nt_user_id' => '1',
'nt_user_session' => '54d9d0d51600b524',
'pass_salt' => '*******',
'self_write' => '1',
'usable_ns' => '2',
'user_create' => '1',
'user_delete' => '1',
'user_write' => '1',
'username' => 'root',
'zone_create' => '1',
'zone_delegate' => '1',
'zone_delete' => '1',
'zone_write' => '1',
'zonerecord_create' => '1',
'zonerecord_delegate' => '1',
'zonerecord_delete' => '1',
'zonerecord_write' => '1'
};
nameservers: 4

Starting import using: /home/pto/home/namedb/gen/named.pto-test.conf
zone: ip.cybercity.dk from gen/11/P.ip.cybercity.dk
creating zone ip.cybercity.dk
Sanity error ( The mailaddr format replaces the @ with a . (dot). ), $VAR1 = {
'contact' => '[email protected].',
'description' => 'imported',
'expire' => 604800,
'minimum' => 300,
'refresh' => 10800,
'retry' => 3600,
'ttl' => 300,
'zone' => 'ip.cybercity.dk'
};
A : lo0.7-tech.ip.cybercity.dk 130.227.88.80
The 'zone_id' parameter (undef) to NicToolServer::Import::Base::nt_create_record was an 'undef', which is not one of the allowed types: scalar
at /usr/local/share/perl/5.14.2/NicToolServer/Import/Base.pm line 124
NicToolServer::Import::Base::nt_create_record(undef, 'zone_id', undef, 'type', 'A', 'name', 'lo0.7-tech', 'address', 130.227.88.80, ...) called at /usr/local/share/perl/5.14.2/NicToolServer/Import/BIND.pm line 105
NicToolServer::Import::BIND::zr_a('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)', 'Net::DNS::RR::A=HASH(0x2125480)', 'ip.cybercity.dk') called at /usr/local/share/perl/5.14.2/NicToolServer/Import/BIND.pm line 65
NicToolServer::Import::BIND::import_zone('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)', 'ip.cybercity.dk', 'gen/11/P.ip.cybercity.dk') called at /usr/local/share/perl/5.14.2/NicToolServer/Import/BIND.pm line 338
NicToolServer::Import::BIND::Conf_Parser::handle_zone('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)', 'ip.cybercity.dk', 'in', 'master', 'HASH(0x178cad8)') called at /usr/local/share/perl/5.14.2/BIND/Conf_Parser.pm line 762
BIND::Conf_Parser::parse_zone('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)') called at /usr/local/share/perl/5.14.2/BIND/Conf_Parser.pm line 856
BIND::Conf_Parser::parse_conf('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)') called at /usr/local/share/perl/5.14.2/BIND/Conf_Parser.pm line 887
BIND::Conf_Parser::parse_fh('NicToolServer::Import::BIND::Conf_Parser=HASH(0x1aef4d8)', 'GLOB(0x1aef490)') called at /usr/local/share/perl/5.14.2/NicToolServer/Import/BIND.pm line 46
NicToolServer::Import::BIND::import_records('NicToolServer::Import::BIND=HASH(0xdbcd40)', '/home/pto/home/namedb/gen/named.pto-test.conf') called at /usr/local/nictool/server/bin/nt_import.pl line 64
/usr/local/nictool/server/bin
root@amnesio:/usr/local/nictool/server/bin#

named.pto-test.conf:
zone "ip.cybercity.dk" { type master; file "gen/11/P.ip.cybercity.dk"; };

The zone file (gen/11/P.ip.cybercity.dk):
; THIS FILE IS GENERATED, DO NOT EDIT
$TTL 300
@ IN SOA ns1.cybercity.dk. zonec.cybercity.dk. (
2013011003 ; Serial
10800 ; Refresh
3600 ; Retry
604800 ; Expire
300) ; Minimum
@ NS ns1.cybercity.dk.
@ NS ns2.cybercity.dk.
@ NS ns3.cybercity.dk.
@ NS ns4.cybercity.dk.

lo0.7-tech A 212.242.40.51
lo0.sagem A 212.242.40.3

Note: the zone file is very long and i have left some out.

I have tried using the version from git but its currently broken according to issue #73

Hi,

I dont know if this feature exists, otherwise i would like to request it.

We are a T2 ISP and we are trying to integrate NicTool as our DNS management system replacing our legacy scripts bases DNS management with NicTool.

Today we have build scripts which auto generates reverse dns zones (192.0.10.1 0x12345678.customer.test.com) and then update all PTR records if they have A-records (A www.test.com 192.0.0.1 will update the PTR for 192.0.0.1 to match www.test.com).

How would you integrate this part into NicTool?

Im thinking to rewrite the reverse dns zone generator script to write bind zones to a directory, then use nt_import.pl to import the PTR zones. And if changes are made, re-do the steps and overwrite the old data.

Next step could be using the API or database access to lookup all A-records and update the PTR zones with the records.

What do you think? Do you have any plans to support a feature like this?

Thanks,
Per N. Toft
Telenor DK

I have just found an interesting bug, i have a user set up in the system designed to only allow zone record modification, I have no groups configured so all users are in the same default group, however i set this user up to just have zone record and self rights.

Logging in as this user, everything works properly, however when i updated the users password, i got the following in the change log:
Thu Sep 4 14:06:48 2014 modified User dns-admin
changed password, changed inherit_group_permissions from '0' to '1'

There is no visible way to control the users group permissions (as specified by the help text as well) so my best guess is it has assumed the group permissions option (as is default on create) and passed that when the password was updated. Issue is that user now has full admin priv's on the system with every tickbox.

For now i'll work around it by removing self :)

if openssl-devel isn't installed (it isn't, on CentOS minimal installs), the subsequent build of Net::SSLeay will fail with compiler errors (missing headers). I'm not sure where to patch for this; server/bin/nt_install_deps.pl has a section at the top outlining e.g. rsync, mod_perl, apache, etc., but openssl-devel might be too vendor-specific to warrant inclusion here.

when exporting to tinydns, the timestamp can set a "start time" for the record, so it doesn't get published until then.

Have Zone::new_zone automatically rewrite those entries before insertion.

Update the JS to format that value more intelligently

The checkboxes don't reflect what's stored about what nameservers should be used.

The problem is probably related to the change in how nameservers are stored.

Fix:

In group.cgi, replace

my %nsmap = map { $data-&gt;{"usable_ns$"} => 1 }
grep { $data->{"usable_ns$"} != 0 } ( 0 .. 9 );

with

my %nsmap = map { $_ => 1 } split(',', $data->{"usable_ns"});

Have a look at BIND::Config::Parser to parse the named.conf file, and Net::DNS::ZoneParse to parse the zone files. Then it should be a small matter to extend NicToolServer::Import with a BIND module.

Are you aware that nictool.com is down? Noticed it yesterday but still down today.