Git Product home page Git Product logo

surveys's Introduction

Surveys

File structure is normalized to allow retrieving surveys from the Devographics app (surveyform, results, API...)

|__<surveyId>
|_____files common to all editions (years) of a survey
|_____<editionId>
|________files for this specific edition
|__surveyParamsTable.json: maps URL slug to surveyId and editionId

surveys's People

Contributors

eric-burel avatar kilian avatar leaverou avatar pepelsbey avatar sachag avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

surveys's Issues

Features Missing From CSS: Parent selector vs `:has()`

I'd like to keep and tweak this question for 2021:
https://2021.stateofcss.com/en-US/opinions/#currently_missing_from_css_wins

Parent Selector is a tricky one, because there's now :has(), but it's different from the imagined parent selector as the reverse of the child selector. Options that I can see:

  • Keep asking about Parent Selector and see if it goes down as a result of shipping :has()
  • Drop the Parent Selector option here and add something else (needs to be 8 options)
  • Somehow explicitly ask "did :has() meet the use cases you wanted parent selectors for?" (other question)

Idea: Organizing features by priority, rather than category, with "Ask me about more" functionality

Right now the CSS features we ask about are organized in categories like Layout, Typography etc, and everybody is asked the same set of questions.
There are several discussions about which features to add, and eventually we're not going to be able to add all of them, simply to not make the survey too long and cause fatigue.

However, I'd wager that some people may actually be fine answering a longer survey, and may even like being exposed to multiple different features they may or may not have heard about.

But here's a crazy idea: what if instead of organizing the features in these thematic categories, we ranked them by priority, and initially showed every respondent a subset of N features that we ranked with the highest priority, with a button "Ask me about more features" that would progressively reveal more? Then everything we consider valuable could get in, and the only hard decision would be how to rank it, not whether to include it at all.

The main caveat being self-selection bias: The kinds of people that click "show me more" may not be a representative sample. Maybe they're more into CSS, or have more time in their hands because they are not fully employed or are younger etc. There are many ways in which this simple decision gets you a less representative group. Not sure if this is enough of a concern to not do this though.

Meta: End of 2022 Roadmap

As of October 6, 2022, the State of CSS 2022 survey is open, and the State of GraphQL 2022 survey results have been published.

Immediate Roadmap (Oct 6 – Oct 15)

  • Increase CSS 2022 outreach to broaden the survey audience and make it more representative.
  • State of GraphQL Results marketing

Short-term Roadmap (Oct 15 – Nov 1)

End-of-year Roadmap (Nov 1 - Dec 1)

  • Launch State of JS
    • Outreach efforts
  • State of JS results
    • New chart types?
    • Custom query builder
    • "Fireworks" dataviz

Opinions: Maybe "missing features" should somehow be clarified?

Having a look at the answers for features missing from CSS from last year, many of them actually do exist in one way or the other.

So the question is, do participants of the survey know that they exist (in a specification) and just mean that there is little or no browser support for them? Or don't they know about them?

I don't remember how the answers to this question were structured. If this was a multiple-choice list or if there was a free form field or both.

But maybe people could somehow be informed about it if a feature they mentioned there actually does exist. This might be directly while giving the answers or when presenting the outcome of the survey or both.

Sebastian

Idea: Provide a post survey call to action

What if there was a post survey prompt "What's next? Here is a list of places where people can go to engage in css discussion and discourse"?

Opportunity for direct user engagement or user interviews?

Idea: Allow respondent to priority rank questions such as Library Evaluation

Hi, writing surveys are tough and often thankless work. So, firstly, thanks for putting together such an amazing survey! The JS and CSS surveys are interesting and important.

Since I'm an unknown to many of you and not involved in your respective communities, hopefully you'll go easy on me if my suggestion is completely out of place. I read through the other open issues, but didn't see anything that captured the following idea.

A stacked list would likely allow for a more accurate expression of the respondent's priorities. The current UI introduces bias by presenting a tournament bracket style UI element. For example, how would a respondent indicate that their top priorities are user experience, documentation, inclusivity?

Tournament brackets also has the implication of a victor, head-to-head competition, and confrontation. In a passionate space such as front-end, this implication moves the narrative further away from inclusivity.

Anyways, nice to meet y'all,
Kind regards

Screen Shot 2022-08-19 at 11 01 33 AM

Meta: Browser incompatibilities, CSS pain points and features missing from CSS overlap

Browser incompatibilities, CSS pain points and features missing from CSS overlap in regard of missing browser support.

What I mean with that is that participants may interpret missing support for a feature as a browser incompatibility. It may also be a pain point for them if they can't use a feature because of missing support. And if a feature is defined in the specifications but having zero browser support it may be feature they are missing from CSS.

So, maybe there could be a separate question asking people for features they know about but are not having enough browser support yet.

This goes in the same direction as #31 with the difference that it also covers features that are supported in browsers but not everywhere yet like Subgrid or :has().

If such a question is added, it should be clearly distinguished from the other three.

Sebastian

State of JS 2022 Preliminary Discussions

How can we improve the 2022 edition of the survey?

See also previous discussion here: Devographics/Monorepo#61

Things to Consider

  • Sections to remove/add?
  • Features questions to remove/add?
  • Libraries questions to remove/add?
  • Other questions to remove/add?
  • Options to remove/add inside specific questions?
  • Question types to change?
  • Other improvements?

2021 Edition for Reference

General Resources

State of CSS 2022 Questions: help us create the survey outline

This year we are very lucky to have @LeaVerou participate in the survey design process. You can review the initial survey outline we came up with together here (new questions are marked with a 2022 tag):

And find the YAML file here:

Note that this is a starting point, and we value any feedback/suggestions about what to change, keep or remove.

What's New

An overview of the main changes so far:

  • Removed the Pre-processors category as it feels like there isn't too much debate around that area.
  • Got rid of "which browser do you primarily develop in?" question as we already ask which browsers people test in.
  • Merged "Opinions" and "Environments" sections into new "Usage" section.
  • Moved browsers question to "Other Tools".

New Features

  • currentcolor
  • color_mix
  • wide_gamut_colors
  • scroll_behavior
  • scroll_padding
  • font_palette
  • focus_visible
  • has_selector
  • where_selector
  • cascade_layers
  • houdini_paint_api

Under Consideration

Some new CSS features (such as CSS nesting) were considered but are not currently included due to the lack of a MDN page, CanIUse page, or just very limited browser support (which are not necessarily disqualifying factors, but are still indicators that it might be too early to include a feature in the survey).

  • object_view_box
  • viewport_percentage_length_units
  • relative_colors
  • color_fonts
  • houdini_typed_om
  • houdini_layout_api
  • houdini_animation_worklet
  • if/when_else_rules
  • trigonometric_functions
  • stepped_value_functions
  • css_nesting
  • env_function

To Do

Still to do:

  • Design a question to measure people's amount of JavaScript usage – maybe a slider between JS and CSS so people can adjust which percentage of each language they spend their time writing?

Related

See also this thread discussing improving the UI for features/libraries questions.

Feature: Wide gamut colors

Wide gamut colors are lab(), lch(), oklab(), oklch() and the color() function

Or should we ask about device independent colors (lab(), lch(), oklab(), oklch()) and wide gamut colors (color()) separately?

(split off from #1 and #2)

Feature: `forced-colors` media feature

I noticed the survey for 2022 has forced-colors commented out and marked "not supported yet". While true on mobile except for Firefox, on Windows support is pretty good. Chrome, Edge and Firefox all have support.

We use this feature in production for some reverts to restore borders1 and to correct an SVG color bug in Chromium.

Maybe the feature is ready for the 2022 survey?

Footnotes

  1. we use box-shadow for the visuals outside of forced colors mode

Author pain points with Custom Properties (and variables)

Not sure if this requires a particular question or not, but I think that as usage of Custom Properties has been ticking up, it might be time to ask for the specific pain point people have noticed in their usage. We know that power users have had issues, but I wonder if these have been noticed by more users as well.

Feature: `:host`, `::slotted()`, `::part()`

:host, ::slotted(), and ::part() are integral to develop/consuming/understanding styling for elements with Shadow Root. They also have a lot of opportunity to be leveled up with recent advances like :has() and @container. Understanding the current state of their usage and any specific issues that CSS developers might be facing with them would be amazingly beneficial in outlining paths forward with them.

Should we ask about HTML features?

The current survey asks about tabindex and ARIA. These don't seem to fit the general CSS theme, and seem a little out of place. Should we continue to include them?

Relevant: would it make sense to have an HTML survey? (could ask about web components too)

(Split off from #2)

Feature: `image-set()` function

Spec: https://drafts.csswg.org/css-images-4/#image-set-notation
MDN: https://developer.mozilla.org/en-US/docs/Web/CSS/image/image-set

This function is supported in all browsers using the -webkit- prefix. Gecko is currently the only one that additionally supports the unprefixed version. The prefixed version is very restricted in what it supports.
Nonetheless, it might be interesting to see how many people are already using it.
In my opinion, this is a relative important but probably underrated feature in regard of UX.
So maybe the survey could also raise the awareness of the function and push browser vendors to implement the unprefixed version.

Sebastian

Editor and tool incompatibilities or issues

In my opinion CSS is moving very fast lately (nice 🎉 ) and because of efforts like interop 2021 these new features ship in all browsers at roughly the same time.

I would like to avoid that tools and code editors start to lag behind on critical features.
This can give the false impression that a feature isn't ready or safe to use.

A good example is @layer which is unusable in VSCode without a bunch of warnings.

Can we ask users which features are causing the most issues in popular editors and tools?

Meta: Asking about developers' experience with each CSS Feature

There is currently a lot of discussion about this scattered in a variety of places, and I wanted to have a specific issue about this.

The problem is that the current survey tracks primarily awareness (Haven't heard about it, Heard about it, Used it), but it does not track what developers think about each CSS feature.

  • "Heard about it" could mean that they just haven't had a chance to use it, or they have no use cases, or they don't see the point because browser support is so bad, or the feature seems confusing and hard to learn, or several other things.
  • "Used it" could mean they used it and loved it and plan to use it in every project now, or they found it very confusing and plan to only use it if absolutely necessary, or they loved it but don't plan to use it again for a while due to browser support etc etc.

There are several thoughts about capturing some or all of that.

Option 1: Ask a followup question for every "Heard about it" and "Used it" answers.

There is a lot more brainstorming about this in Devographics/Monorepo#99 but I believe the latest mockup looks like this:

image

Pros: Captures pretty much all of the nuance described above
Cons: Increases cognitive overhead for each feature question quite significantly.

Option 2: "Would use it again" / "Would not use it again" distinction

There are two ways to implement this:

  • a) Either break the "I have used it" answer into two: "I have used it, would use again" and "I have used it, would not use again"
  • b) Or have a popup similar to the above, but with a simple radio: "Would you use it again? 🔘 Yes 🔘 No". Note that we cannot use a checkbox here, as the default will affect the results, both Yes and No need to be a deliberate choice.

Pros: Much lower cognitive overhead than Option 1, especially with Option 2a.
Cons:

  • The obvious con is that we get much less information, and none about the cases where people have only heard about a feature (but may still have opinions!). The idea being there can always be followup surveys about specific features that score poorly.
  • Option 2a breaks historical comparison. We could still group them together for that, but the mere act of having two answers instead of one affects what respondents do. While this is a downside, it's not necessarily a dealbreaker.
  • Option 2b is still in essence a separate question, so there's still some of the cognitive overhead of that, while getting a lot less info.

Option 3

Emoji rating widget

Other ideas

Perhaps the best way forwards is a combination of 1 and 2. E.g.: a followup popup for "Would you use it [again]?" ("again" only for "I have used it" answers) with an "Add details" UI.

Survey Design Credits

@djaddison @SebastianZ would it be ok to credit you for the CSS 2022 survey design? If so could you give me links to your Twitter accounts? (we pull in avatar, etc. from the Twitter API).

Meta: Should the question about usage of a feature be more precise?

When I participated in the past surveys, I sometimes didn't know whether to choose "Know about it" or "Have used it".

As a CSSWG member and CSS enthusiast, I almost know all of the CSS features and try them out as soon as they are available. Though many of those things didn't find their way into production code at that point. So it feels a bit weird to say that I used a feature when I was just testing how it works.

So maybe an explanation of the options would already help in that situation.
There might also be three options for "having heard/read about it", "having tried it out" and "using it on a website".

Sebastian

Idea: Provide more survey context upfront

Some things that would have helped me when engaging with the survey would have been:

  • What is the intention or goals of the survey? (this becomes even more important for new devs, first timers, or people unfamiliar with the css community)
  • What is the impact of the results?
  • Who is reading the results of the survey?
  • Does this information inform the community, standards organizations, developers, tool implementers, etc?

What would the survey look like if there was a "Before you start" page that provided some initial context?
Additionally, the content of the FAQ is small, would it be more approachable if it wasn't hidden in an accordion?

Feature: Two-value syntax for `display`

Spec: https://drafts.csswg.org/css-display/#the-display-properties
MDN: https://developer.mozilla.org/en-US/docs/Web/CSS/display
Other links:
https://css-tricks.com/two-value-display-syntax-and-sometimes-three/
https://hacks.mozilla.org/2019/10/the-two-value-syntax-of-the-css-display-property/

This is probably not commonly used yet, though once this got implemented in Gecko, Rachel Andrew and Miriam Suzanne tried to raise the awareness of there being an inner and an outer display.
So maybe the survey can achieve that goal and push the other implementors to also pick up this feature.

Sebastian

Better Survey Outreach

How could we reach beyond the survey's current audience and make sure we include a more representative audience? I thought I'd list some ideas here.

  • Advertising
    • Reddit
    • LinkedIn
  • Blog posts
    • Smashing Magazine
  • Interviews/AMAs
  • Reach out to influencers directly
  • Reach out to newsletters

Other suggestions welcome!

New Question Idea: "Obsolete Features"

It could be fun to ask about obsolete features or patterns. Things like table layouts, float layouts, the <font> tag… I don't know if there would be enough of these to make for a full question but I'd be curious to know what proportion of our audience is old enough to have used them!

Widely used CSS features to remove

There have been plenty of excellent suggestions already, so we should consider what we could remove to make space for adding questions about new features.
Issue #11 already discusses removing the HTML questions, and #10 discusses removing the Houdini JS API questions.
However, there are a few CSS features that are very widely used and we don't gain very much insight asking about them.

The features with > 90% usage in 2021 were:

  • Flexbox
  • calc()

So it seems reasonable that these could be removed. Are there any others that could be safely removed?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.