The API should have a way to enable/disable Pi-hole.

Update Rust to the latest nightly to take advantage of Rust 2018 and Rocket 0.4.
Rust 2018: https://blog.rust-lang.org/2018/12/06/Rust-1.31-and-rust-2018.html
Rocket 0.4: https://rocket.rs/v0.4/news/2018-12-08-version-0.4/

Provide the versions (and branches) of various Pi-hole components, including:

• Core
• Web
• FTL
• API

Possible format:

{
    "core": {
        "version": "v4.0",
        "branch": "master"
    },
    "web": {
        "version": "v1.0",
        "branch": "master"
    },
    "ftl": {
        "version": "v4.0",
        "branch": "master"
    },
    "api": {
        "version": "v1.0",
        "branch": "master"
    }
}

Pretty simple feature. Core at the moment is still based on git, so we will get the version from git. This is the way that the updatechecker.sh script currently gets the version:

git describe --long --dirty --tags

But there may be a more programmatic way to get this without running a raw git command.
Web's version will probably be encoded inside of the API, since web will be embedded inside of the production binary.
FTL can be gotten with the command >version sent through the FTL API socket.
And the way to get the API's own version is obvious.

This is a sub-issue from #7 focusing on writing settings. This should be completed after #28.

Settings should be exposed in some RESTful fashion (see #28). They can accept various HTTP verbs such as PUT to update the setting.

The incoming data to be written should be validated before storing, and will require new error types.

The PHP web interface validates and modifies the settings through the savesettings.php file, which is all backend code. The actual modifying of the settings is delegated to a Core script: webpage.sh. This made testing the code pretty hard (it's not actually tested).

The block page was originally provided via a PHP file in the Core repo which acted as the 404 page for lighttpd. The block page is only used for IP-based blocking, which is not the default anymore (replaced by NULL).

The block page should be implemented in the API via a server-side rendered template. This is supported by Rocket via Handlebars or Tera: https://rocket.rs/v0.4/guide/responses/#templates

The block page is not a hard requirement for alpha or beta, but should be implemented by the full release so we maintain feature parity.

Relevant SetupVars entries:

• API_EXCLUDE_DOMAINS
• API_EXCLUDE_CLIENTS

A test function like this should have #[cfg(test)] set on it - it shouldn't be usable from normal configurations.

Is this project still active? The last commit was made 8 months ago. There is also no documentation for this project which makes it pretty difficult to use.

Add a way to query long term data for stats. See the PHP API for how it handles this: https://github.com/pi-hole/AdminLTE/blob/master/api_db.php

Add a way to query long term data for display on the web interface dashboard (summary stats, queries over time, top lists, etc). This has been split out from #40.

Create a way to authenticate the API user. Most endpoints will be protected. The authentication will be session-based like the current PHP web interface. The security can potentially be tighter as well, but that may be improved after the initial authentication is implemented.

This story will be developed alongside web.
Corresponding issue: pi-hole/web#10

This can be done by creating a Request Guard.

An example of a session-based authentication is found in one of Rocket's tests.

While playing around with this API on my own Pi-hole (v4.1.1), I am struggling to find documentation for the different endpoints and their options.

Currently I am using the blogpost (https://discourse.pi-hole.net/t/pi-hole-api/1863), but this seems somewhat outdated.

Is there any place I can find up-to-date or version-based endpoint documentation?

If SetupVarsEntry::WebPassword is empty, we still treat that as a password hash. However, that prevents anyone from logging in via the web interface.

If the password is empty, we should allow unauthenticated access to endpoints which require auth.

We should also signal to the web interface in some way that it should skip the logged out state (immediately set loggedIn to true).

Add support for /stat/history pagination for pi-hole/web#57.

The pagination cursors should be stable as the query history is expanded (old queries are removed after 24 hours and new ones are added as they are made). The cursors can be based off of the query IDs in the database to accomplish this.

Testing can always be improved. In some cases, only the "happy paths" are tested, not the cases where an error should occur or some other edge case. Also, there are a lot of integration-type tests over a single endpoint, but there should be more unit tests on specific functions. Right now there aren't too many missed spots.

Support for adding and removing DHCP leases is required so the web interface can expose that functionality. Additionally, the API should expose the currently active leases.

Will this new API eventually replace the current PHP API in AdminLTE, or will they both exist?

In order to support the settings page of the web interface, we need some way to read, validate, and modify Pi-hole settings.

Reading

See #28

Writing

See #29

The goal is to have equal or better handling of settings validation and modification.

Add a configuration file to configure the API.

Some options might be the address and port it binds to, logging verbosity, custom location of files like setupVars.conf or whitelist.txt, etc.

The config will probably be a TOML file, since they are simple to use and are very well supported in Rust.

I would like to add an LDAP authentication.
Few questions:

1. Is auth key currently hashed on client side (react)? If true do you plan to replace it with plaintext + https option? I do not think it is possible to do client sode hashing with ldap.
2. Is there an option secure the whole website and not just a few pages?
3. Why do you keep web and api projects separate? It does not seem to simplify branch development model as it usualy covers both.
4. Is it OK if I add login field to the web ui if ldap is enabled?

We currently lack insight into test coverage. We need to look into test coverage tools so we can measure how well the code is tested.

The recommended service to use is Codecov (example docs). It can be supplied data from kcov (see the example docs) or tarpaulin.

Supports #30

failure is a crate for error management. It allows you to easily create errors with all the functionality you need, including backtraces, associated data, and error messages. Its biggest strength though is that these errors can be used alongside every other error, so you can set the output of a function to Result<T, failure::Error> and every kind of error can be thrown without losing information like the backtrace and cause.

When a list file, such as whitelist.txt does not exist, the API throws an error.
This is a valid case, because some of the files are not created during the install. If they are missing they are assumed empty. The code was made to handle this, but some refactoring may have broken it.

All operations (GET, POST, DELETE) on the whitelist and blacklist have this issue. The regex list does not, because it is created during install.

Hi,

First off, thanks for all your hard work on this project !!

Second, I have a small feature request - if would be great if I could get by resolver in the query log.

I would like to see which lookups was sent to which resolver.

Thanks

Shared memory requires that both sides use identical structures, otherwise the shared memory could get corrupt or be read incorrectly. To prevent these issues, there should be a field in shared memory which records the version it is at, like a database version. Then everyone who connects to the shared memory can validate that they are using the correct version of structures.

This will require changes in both FTL and API, but should be trivial. FTL should expose a new shared memory file which will contain configuration details. The only configuration detail at first will be the version, but it should have room to grow if necessary. To allow this growth, the version must be the first item in the shared config structure, and it must have a fixed size (eg. can't change from u8 to u32 when a new version is released).

FTL is implementing a way for it and the API to share stats memory. This will eliminate the stats functions from the FTL socket API, and that processing can now happen in this API.

Each major FTL structure is in a separate shared memory mapping, and there is a string buffer map which holds the strings used.

The API will connect to these shared memory files and directly access the data to respond to API requests. There will be a way for the API to take a read lock on the structures, to avoid incorrect data.

Currently, the FTL code is in the new/shmem branch.

In a private prototype, I have found that the shmem crate is the best option for connecting to the shared memory. The shared_memory crate was a runner-up, but since we are not connecting to shared memory that it created, its locks and events are of no use, and it always opens the shared memory in read/write mode.

Currently, the API is compiled with too new of a glibc version. A fix is to change the build system to use CentOS for building. This would make sure any distro that's at least as new as CentOS can run the API.

If this is done, the DEB build would have to happen in a different step, like how the RPM build is currently.

I've encountered an issue adding a domain to the whitelist. The entry is added to the list despite the error - it is in the GET response and in the php output of /admin/list.php?l=white .

Reloading the lists manually works without error:

pihole -g --skip-download --whitelist-only
[i] Pi-hole blocking is enabled
[i] Using cached Event Horizon list...
[i] 112317 unique domains trapped in the Event Horizon
[i] Number of whitelisted domains: 3
[i] Number of blacklisted domains: 0
[i] Number of regex filters: 0
[✓] Parsing domains into hosts format
[✓] Cleaning up stray matter
[✓] Force-reloading DNS service
[✓] DNS service is running
[✓] Pi-hole blocking is Enabled

Version:

{
    "api": {
        "branch": "HEAD",
        "hash": "d3aef9a",
        "tag": ""
    },
    "core": {
        "branch": "development",
        "hash": "5950b4b",
        "tag": ""
    },
    "ftl": {
        "branch": "tweak/further_general_improvements",
        "hash": "671ca07",
        "tag": ""
    },
    "web": {
        "branch": "development",
        "hash": "e554991",
        "tag": ""
    }
}

Reponse:

{
    "error": {
        "data": null,
        "key": "gravity_error",
        "message": "Failed to create the blocklist"
    }
}

If I submit the request again - we get an item already exists reponse.

This could be due to how I have the API installed - I noticed that prevous to this I would get access issues to whitelist.txt and blacklist.txt such as:

{
    "error": {
        "data": {
            "file": "/etc/pihole/whitelist.txt"
        },
        "key": "file_write",
        "message": "Error writing to /etc/pihole/whitelist.txt"
    }
}

These were resolved by changing owner and group on the files to 'pihole'.

Any questions about this let me know.

Thanks,

James

When we ship the API and Web interface, we want them to be in the same binary. The API will host the web interface under /admin and the API endpoints under /admin/api.

Obtaining Web Interface Builds

To fit them in the same binary, we first need to have some way of specifying and obtaining built versions of the web interface. We could host the built web interface under ftl.pi-hole.net like we currently do with FTL and the API, but there's probably a better way.
It might be possible to have the web interface upload the built version to CircleCI as a Build Artifact and then download it during an API build.

Packaging the Web Interface Into the Binary

Once we have the web interface files, we need some way to store them in the binary so we can access them again. There is a library we can use to easily accomplish this:
https://crates.io/crates/rust-embed
It looks decent, and was built for this exact use case:

You can use this to embed your css, js and images into a single executable.

There are even examples of using it, including an example for Rocket.

This seems to be the easiest way to embed the web interface into the API binary.

We are working out the details of moving the various lists we utilize into a database, including gravity, adlists, whitelist, blacklist, regex list, etc. The API will need to go through the database to access this information.

This should be done after #40, because that issue will work out the kinks of adding a database to the API.

This is a sub-issue from #7 focusing on reading settings.

Settings should be exposed in some RESTful fashion, such as /admin/api/setting/upstream_dns. This issue only focuses on reading the setting (HTTP GET).

The settings are stored in /etc/pihole/setupVars.conf by default (see PiholeFile::SetupVars). There is already a method called read_setup_vars in setup_vars.rs which can read data from the file, but it might need to be changed into a more structured form. This form should easily be able to be extended for writing settings (see #29).

I am using proprietary version of centos,hence there no provision of yum install as it is my NAS server.

[/] # yum
-sh: yum: command not found

--------------------------Query to be asked---------------------------

1.Is there any other way to install .rpm package when rpm is unsupported ?
2. Or How can i install rpm ?

[/] # rpm
-sh: rpm: command not found