headers has a text_utf8 constructor for ContentType. I suggest a html_utf8 constructor be added, as well, resulting in

Content-Type: text/html; charset=utf-8

Motivation: If the charset is not set for the text/html content type, then it defaults to ISO-8859-1, but in many situations, the charset is really UTF-8.

Some headers use a single integer to represent time in seconds, like:

• Retry-After: 300
• Access-Control-Max-Age: 6000
• Cache-Control: max-age=6000

Should the value be exposed as a Duration instead of just a u64? That may make it clearer what the value actually means.

There could be a sealed trait used for the constructors that is implemented for u64 and Duration, so that people can use either (in case someone really wants to say RetryAfter::new(300) instead of RetryAfter::new(Duration::from_secs(300))).

The documentation for the crate does not specify what is the common way to use this crate - it only explains how to extend it for new headers.

It took a bit of trial and error until I finally found the trait HeaderMapExt. And even that trait isn't documented.

The documentation should have something like:

use headers::HeaderMapExt;

let mut headers = HeaderMap::new();
headers.typed_insert(Upgrade::websocket());
let ws = headers.typed_get::<Upgrade>();

The inner field of SetCookie is private, so it can't be constructed by the users of the crate.

The Cache-Control header is just interesting enough that I feel it isn't quite obvious which of the following designs make more sense.

List of Directives

This is how it was in hyper 0.11.x. Essentially, it's just a comma separated list of CacheDirectives, and not any smarter than that.

pub struct CacheControl {
    directives: Vec<CacheDirective>,
}

Example construction:

let cc = CacheControl::new(vec![
    CacheDirective::NO_TRANSFORM,
    CacheDirective::max_age(Duration::from_secs(300)),
]);

This matches the spec more directly, as its ABNF claims to just be a list of directives, and clients may not understand newer directives. However, with the more conservative API (to allow adding new directive support without being a breaking change), the CacheDirective::Ext is no longer exported, so servers/clients couldn't really use "newer" directives anyways, without updating this crate.

The downsides are that this results in an extra allocation of a Vec, and is more complicated for checking if a certain directive is set (one has to iterate looking for a specific directive).

Set of Fields

This would be a plain old Rust object with fields representing known directives.

pub struct CacheControl {
    no_store: bool,
    no_cache: bool,
    max_age: Option<Seconds>,
    // ...
}

This would mean that an allocation of a mostly useless Vec can be removed, and accessors like cc.is_no_cache() can be added cheaply.

The struct could be made even smaller by using bitflags for all the boolean fields.

Lazy (FlatCsv)

Some other fields are currently just lazily parsed when they would otherwise be a list of strings. This saves on allocations/copies (and makes encoding super cheap) at the expense of needing to parse on any lookup. For CacheControl, it seems feasible that you'd need to check for various directives. You could do so once in a single for dir in cc.iter(), but subsequent ones would repeat all the work.

I think I'm leaning towards the Set of Fields option.

This would be tricky, since CSP header values are quite tricky, but also since they're tricky, they would be nice to have typed constructors for.

Right now, Cookie and SetCookie are just strings. Should they expose the various attributes (e.g. Expires, Domain &c.)?

I'm trying to define a typed header for X-Forward-For, but I can't get around the const NAME: &'static HeaderName requirement which only works with the standard headers (to my knowledge).

Could/should this switch the Header trait to a name() method instead of an inner constant?

Having version numbers such as 0.0.3 for headers-ext means that you can never release a non-breaking change to that crate (0.0.z isn't considered compatible with any other version by cargo). Could you make new releases with at least 0.1.0 for the version numbers?

Cc @Eijebong

Currently all typed headers must copy and allocate their own storage for any string data it needs to store. This seems a bit wasteful in common headers such as Cookie where we should really just be able to borrow the existing value instead of copying the entire thing.

I'd like to propose a small change to the Header trait such that it takes a lifetime which allows the decode method to be implemented in a way that borrows the HeaderValues taken as argument.

pub trait Header<'value> {
    /// The name of this header.
    fn name() -> &'static HeaderName;

    /// Decode this type from an iterator of `HeaderValue`s.
    fn decode<I>(values: &mut I) -> Result<Self, Error>
    where
        Self: Sized,
        I: Iterator<Item = &'value HeaderValue>;

    /// Encode this type to a `HeaderMap`.
    ///
    /// This function should be infallible. Any errors converting to a
    /// `HeaderValue` should have been caught when parsing or constructing
    /// this value.
    fn encode<E: Extend<HeaderValue>>(&self, values: &mut E);
}

Drawbacks

• Increased complexity
• Some headers (Cookie) merges the HeaderValue which forces a Cow or the merging must be replaced by storing multiple headers in a Vec/SmallVec instead.

An extension trait to offer something like typed_header on http::response::Builder would be nice.

Several headers that a client can send include a list of "quality items", thereby ranking which sorts of representations are more preferable. For example, Accept: image/svg+xml, image/*;q=0.8. These are often used as part of "Content Negotiation". The API design in this library for these headers should be reconsidered here.

Hi,
With tokio 1.0 and many ecosystem crates depending on bytes 1.0 it'd be great if a new version of headers could be cut (given that master depends on bytes 1.0 now) so that there are less duplicate dependencies around.

The Content-Disposition header is a more complicated one, and coming up with an appropriate design that is easy and clear to use while also being efficient is important.

Where as most of the fields that can be set in CacheControl (#3) are known, and can be represented in bitflags, the fields of a ContentDisposition are typically strings. It'd be useful to be able to set these fields easily (not having to worry too much about converting to a valid HeaderValue), without needing to make several copies when parsing.

One option is to add a ContentDisposition::builder(), which can set the various fields, and the validity can be checked when build() is called.

could you publish a new crate with the change in #72 when you have the time? many thanks

https://rustsec.org/advisories/RUSTSEC-2020-0146.html has started being reported by cargo audit; and the 0.8 sha-1 has a transitive dependency on the vulnerable version of generic-array:

        │   └── sha-1 v0.8.1
        │       ├── block-buffer v0.7.3
        │       │   ├── block-padding v0.1.5
        │       │   │   └── byte-tools v0.3.1
        │       │   ├── byte-tools v0.3.1
        │       │   ├── byteorder v1.3.4
        │       │   └── generic-array v0.12.3  # crate w/ RUSTSEC
        │       │       └── typenum v1.12.0
        │       ├── digest v0.8.1
        │       │   └── generic-array v0.12.3 (*)

I'm using "headers" together with warp to decode basic auth headers. With the current behaviour, my web server would panic if the header value does not start with "Basic ".

It would be great in my opinion if Credentials::decode() could return a Result instead of an Option. In the above use case, Basic::decode() would return an Err instead of panicking.

Current behavior:

let header_value = http::header::HeaderValue::from_static("Bazic xxx");
let basic = Basic::decode(&header_value);  // panic

Feature request (with error handling):

let header_value = http::header::HeaderValue::from_static("Bazic xxx");
let basic = Basic::decode(&header_value).map_err(...)?;  // proper error handling

Forwarded is the standardized counterpart to x-forwarded-*.

Hello,
It would be nice to implement the Age header (used by proxy caches).
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Age

Looking thgroughthe code, it seems that multi-valued headers will just be split at each comma, even if the comma appears inside a quoted string.

Can you clarify whether I am missing something?

See the discussion in tokio-rs/axum#1781 (comment) for the original issue & motivation.

TL;DR is that Header::decode allows each typed header to handle empty iterators in its own way and let the header implementation decide whether missing header is an error or whether to provide some default "parsed" value (e.g. empty collection), but HeaderMapExt::typed_{try_}get always return Option::None when header is not found in the HeaderMap, not giving such Header implementation a chance to correctly provide their own fallback.

It would be useful to have variation of those methods that always return Result<H, Error> by forwarding an empty iterator to the specific H::decode implementation and letting it decide whether to provide a Ok(fallback_value) or emit an error.

No need to have headers re-export headers-ext.

Hi,

Is this draft planned to be supported by the project?
I'd like to implement the Reporting API for the servo project and i need to parse the "Reporting-Endpoints" header.

I wonder if it could be useful to add the parsing and the structured header into this lib?

Any ideas/comments/guidance are welcome :)

Regards

Users don't always need all the functionality,
we should use feature flags to let them choose, thereby reducing compilation time

Per rwf2/Rocket#1067 I looked into reexporting typed headers from this crate, as they were reexported from hyper 0.10 in the past. But I quickly discovered that many headers apparently can't be constructed, because they have a single private field and nothing like a new or from method.

In particular I was trying to use Location and ETag, but given #40 and after looking at the docs it looks like many other header types don't have public constructors either. Am I misunderstanding the purpose of this crate or misreading a trait bound, or are these simply missing constructors that need to be implemented?

I noticed the self_update crate is pulling in a bunch of outdated dependencies through hyper_old_types, but uses only its typed headers for LinkValue and RelationType. I attempted to switch out hyper_old_types for headers, but found that these aren't defined here currently. Have they been removed intentionally or is this crate more of a from-ground rewrite than an extraction of the old typed header from hyper and only needs a PR to add these?

The following test exposes a few issues in headers::authorization::Bearer:

#[cfg(test)]
mod tests {
  #[test]
  fn test_invalid_auth_header() {
    // Malformed, should not parse.
    let value = http::HeaderValue::from_str("Bearer    foo, Bearer bar").unwrap();
    use headers::authorization::Credentials;
    let decoded = headers::authorization::Bearer::decode(&value);
    println!("decoded = {:?}", decoded);
    println!("if decoded succeeded, the token was: {:?}", decoded.as_ref().map(|b| b.token()));
    // The header is malformed, so parsing should fail:
    assert!(decoded.is_none());
  }
}

Basically, Bearer's implementation will strip the leading string "Bearer " from the value, and return the remainder for as a token. This isn't valid according to the grammar for the Bearer authn scheme. (The grammar of the token itself is restricted to a certain set of characters, for example.) In fact, the code only checks that the header starts with Bearer in debug mode (as it is done w/ a debug_assert!), so in a release build, any value will parse, including, e.g., Basic <a basic headers' credentials>.

Even where the header is valid, and should parse, the .token() determines the token by simply removing "Bearer ".len() characters from the front of the header value, which itself is not correct: Bearer foobar (n.b., the double space; this is permitted by the grammar for the header, but is not part of the token) parses as it should, but .token() returns " foobar".

Last, the debug-mode-only debug_assert! asserts that the value starts with Bearer — but auth schemes in HTTP are case insensitive, so this too will reject valid inputs. (Note: the Basic auth-scheme parser shares its own version of this issue, too.)

I was trying to add some explanation on why header parsing fails to a custom typed header and couldn't do so because the error class doesn't have a way to add additional information.

I was thinking one could change the invalid signature to Error::invalid(Option<String>) so that a custom error message could be optionally added when parsing fails?

Would love to hear thoughts on this.

There is an attribute of HeaderValue that marks it as "sensitive". This currently has 2 effects:

• In HTTP2, the HPACK never-indexed-literals flag is set. This keeps the value from being stored in the dynamic HPACK table.
• Alters the Debug output to simply write the word "Sensitive" instead of the actual bytes, which can help with accidental storage of secrets or PII.

Should these potentially sensitive headers set this flag when encoding to a HeaderValue?

• Authorization
• Cookie
• Set-Cookie
• Others?

This algorithm is required to extract a MIME type.

For implementations like Servo that are implementing an HTTP cache, it would be nice to reuse the ETag machinery that already exists in this repository. Right now it's impossible to do anything with the typed ETag header and we need to reimplement the weak/strong comparisons.

the following implementation adds auth headers to a header map:
impl<C: Credentials> ::Header for Authorization<C> {

The inserted header should be marked sensitive to not occur in any logs

https://fetch.spec.whatwg.org/#origin-header has a very strict ABNF. Strings ending in # or / shouldn't be allowed. Because OriginOrNull processes strings via url parsing (

According to https://github.com/web-platform-tests/wpt/blob/master/cors/remote-origin.htm uppercasing the scheme as HTTP: should likewise be considered invalid, although that's not as obvious from the grammar.

The currently published headers (0.3.4) is affected by RUSTSEC-2020-0071 , because it depends on time 0.1. This is already fixed on master by 4bf199f, and a new release with this fix would be appreciated. Thanks! 🙂

The Connection header lets me test if it contains specific "connection options"¹ are contained in it. But it doesn't let me actually iterate its contents to see what it contains.

This is very frustrating as I need to actually know what's in the header because I'm implementing a gateway and I need to strip the referenced header names from the request I'm proxying. I would really like to be able to use headers::Connection so I don't have to worry about parsing the comma-separated list myself.

In theory I could just parse the header and then walk the whole headers map and compare each entry to the Connection, except HeaderMap does not offer a retain() operation and so that approach would require reallocating the whole map even if I don't end up making any changes.

tl;dr: bytes=-x is being mapped to ..x by this crate, where http headers mean the last x bytes, and rust BoundRanges mean the first x bytes.

Introduction to the ranges and their definitions

HTTP Ranges

For HTTP ranges, there are four options:

1. Not defining a range. This means everything is included
2. Range: <unit>=<range-start>-, where only the range beginning is given. This means everything from start is included.
3. Range: <unit>=<range-start>-<range-end>, where both the start and end is given. This means everything between start and end are included
4. Range: <unit>=-<suffix-length>, where just a suffix is given. This means only the tail, the last suffix bytes are returned.

Rust RangeBounds

Rust's RangeBounds also have (kinda) four options:

1. .., which includes everything.
2. start.., which includes everything except the first start bytes.
3. start..end (or start..=(end - 1)), which includes everything from start to end, excluding end.
4. ..end (or ..=(end - 1)), which includes everything from 0 to end, excluding end.

Current Mapping between them

Currently, this mapping is just done one-to-one based on their syntax, not based on their definition. This means that Range: bytes=x-y is mapped to x..y, which is fine if both x and y are defined. Only including x is also fine, as they'll still mean the same. But, if only have y, the mapping becomes wrong.

There is no correct mapping for these cases, as "give me the last n bytes" is not something you can represent in Rust's RangeBounds. When the length we're dealing with is known, conversion is possible, but RangeBounds are probably not the right type in that case for storing the header information, as converting back isn't possible.

Options

Require length before returning ranges

Given the length, converting from HTTP Ranges is possible. This would mean that the signature of headers::Range::iter would need to be changed, to include the length of the content being returned.

Don't use RangeBounds

Alternatively to giving the length, we could use a separate type here, and have a conversion method (which takes the length again) to map it to RangeBounds.

I'm admittedly new to Rust so correct me where I'm wrong but it appears to me that a user is only able to construct a Vary: * header and is not able to construct a "custom" Vary header like Vary: accept-encoding.

If I'm correct, I'd be happy to submit a PR that enables creation of custom Vary headers though I'd probably need some direction from maintainers on what the API should look like.

This a note that I started working on WWW-Authenticate in case someone else does, too.

This issue is to discuss the core Header trait design.

It's currently this:

pub trait Header {
    /// The name of this header.
    const NAME: &'static HeaderName;

    /// Decode this type from a `HeaderValue`.
    fn decode(values: &mut Values) -> Option<Self>
    where
        Self: Sized;

    /// Encode this type to a `HeaderMap`.
    ///
    /// This function should be infallible. Any errors converting to a
    /// `HeaderValue` should have been caught when parsing or constructing
    /// this value.
    fn encode(&self, values: &mut ToValues);
}

/// An iterator of `HeaderValue`s supplied to `Header::decode`.
#[derive(Debug)]
pub struct Values<'a> {
    // ..
}

/// A builder to append `HeaderValue`s to during `Header::encode`.
#[derive(Debug)]
pub struct ToValues<'a> {
    // ..
}

impl<'a> ToValues<'a> {
    /// Append the `HeaderValue` to the existing list of headers.
    ///
    /// While this can be called multiple times, *most* headers should only
    /// call this once. The exceptions are outliers like `Set-Cookie`.
    pub fn append(&mut self, value: HeaderValue) {
        // ..
     }

    /// Append the `impl Display` to the list of headers.
    ///
    /// # Panics
    ///
    /// Encoding `HeaderValue`s is expected to be infallible. However, not
    /// all UTF-8 sequences are valid for a `HeaderValue`. The type passed
    /// here must ensure that its resulting string is a valid `HeaderValue`.
    pub fn append_fmt<T: fmt::Display>(&mut self, fmt: T) {
        // ..
     }
}

Is this a good design?

Please add typed support for Content-Language headers.
I saw there is some implementation in src/disabled
but I assume this was not specific enough?

Hello, Thanks for crate.

It would be helpful, if util submodule is made public, so that one can easily create their own typed headers, using them.

Current code allows creation of an HSTS header, but no access to its contents.
include_subdomains and max_age are private. Created servo/servo#25404 as temporary hotfix.

If the scheme is http and the port is 80, or https and 443, the port should not be included by default. Per testing in servo/servo#22090 (comment) this is not web compatible.

Hello,

I'm packaging headers for Fedora and I noticed that it uses sha-1 which depends on quite some number of crates which had last release in 2016/2017 and we try to avoid packaging such crates. Would it be feasible to swith to sha1?

Thanks in advance!

According to CORS:
Access-Control-Allow-Headers: "Access-Control-Allow-Headers" ":" #field-name
You can find the definition of the construct #rule in the http1.1 rfc.

Then why calling iter() on such a header: "AccessControlAllowHeaders(",x-print") (note the comma) returns an empty iterator instead of something like "", "x-print" ?

This behavior was implemented by 026949a but is not consistent with all the other header types having a #field-name as values (AccessControlExposeHeaders for instance).

According to to the quoted RFC 6265, section 4.2.2

Although cookies are serialized linearly in the Cookie header,
servers SHOULD NOT rely upon the serialization order. In particular,
if the Cookie header contains two cookies with the same name (e.g.,
that were set with different Path or Domain attributes), servers
SHOULD NOT rely upon the order in which these cookies appear in the
header.

The relevant function Cookie::get does not comply with that:

Instead, it only takes the first cookie value of a certain name.

The documentation should call this out, especially if this is not a de-facto standard somewhere because then it can create security vulnerabilities with different parts of a web stack taking different (first, last) values of a cookie with a certain name as authoritative.

Hi! Thanks for your work on this crate! I took it for a spin, and wanted to report a bug I encountered on HEAD of master.

Given the following Rust code:

use headers_derive::Header;

#[derive(PartialEq, Debug, Header)]
struct RequestId(String);

fn main() {
    println!("Hello, world!");
}

...I get the following compiler error:

error[E0433]: failed to resolve: could not find `util` in `{{root}}`
 --> src/main.rs:3:28
  |
3 | #[derive(PartialEq, Debug, Header)]
  |                            ^^^^^^ could not find `util` in `{{root}}`

error[E0425]: cannot find value `REQUEST_ID` in module `__hc::header`
 --> src/main.rs:3:28
  |
3 | #[derive(PartialEq, Debug, Header)]
  |                            ^^^^^^ not found in `__hc::header`

error[E0277]: the trait bound `_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue: std::convert::From<&std::string::String>` is not satisfied
 --> src/main.rs:3:28
  |
3 | #[derive(PartialEq, Debug, Header)]
  |                            ^^^^^^ the trait `std::convert::From<&std::string::String>` is not implemented for `_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue`
  |
  = help: the following implementations were found:
            <_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue as std::convert::From<&'a _IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue>>
            <_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue as std::convert::From<_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderName>>
            <_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue as std::convert::From<i32>>
            <_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue as std::convert::From<u32>>
          and 6 others
  = note: required because of the requirements on the impl of `std::convert::Into<_IMPL_HEADER_FOR_REQUEST_ID::__hc::HeaderValue>` for `&std::string::String`

error: aborting due to 3 previous errors

Some errors occurred: E0277, E0425, E0433.
For more information about an error, try `rustc --explain E0277`.
error: Could not compile `headers-repoduction`.

To learn more, run the command again with --verbose.

I've got a minimal reproduction of this here. I've been able to reproduce this on the 2018 and 2015 editions with the same error.

Put it in headers-ext instead, keeping just the Header trait as the stable core.

#25 (comment)