Git Product home page Git Product logo

safelinks's Introduction

safelinks

Go-based tooling to manipulate (e.g., normalize/decode) Microsoft Office 365 "Safe Links" URLs.

Latest Release Go Reference go.mod Go version Lint and Build Project Analysis

Table of Contents

Project home

See our GitHub repo for the latest code, to file an issue or submit improvements for review and potential inclusion into the project.

Overview

This repo is intended to provide various tools used to manipulate (e.g., normalize/decode) Microsoft Office 365 "Safe Links" URLs.

Tool Name Overall Status Description
usl 🆗Beta Small CLI tool for decoding a given Safe Links URL.
dsl 💥Alpha Small CLI tool for decoding Safe Links URLs within input text.
dslg 💥Alpha GUI tool for decoding Safe Links URLs within input text.
eslg 💥Alpha GUI tool for encoding normal URLs within input text for testing purposes.

Features

usl CLI tool

Small CLI tool for decoding a given Safe Links URL.

  • Specify single Safe Links URL
    • via positional argument
    • via flag
    • via interactive prompt
  • Specify multiple bare Safe Links URLs (no surrounding text)
    • via interactive prompt
    • via standard input ("piping")
    • via file (using flag)
  • Optional verbose listing of query parameter values within a given Safe Links URL.

dsl CLI tool

Small CLI tool for decoding Safe Links URLs within surrounding input text.

  • Specify single Safe Links URL
    • via positional argument
    • via flag
    • via interactive prompt
  • Specify multiple Safe Links URLs (with surrounding text untouched)
    • via interactive prompt
    • via standard input ("piping")
    • via file (using flag)

dslg GUI tool

GUI tool for decoding Safe Links URLs within given input text.

  • Specify single Safe Links URL
  • Specify multiple Safe Links URLs (with surrounding text untouched)

eslg GUI tool

GUI tool for generating test data.

Changelog

See the CHANGELOG.md file for the changes associated with each release of this application. Changes that have been merged to master, but not yet an official release may also be noted in the file under the Unreleased section. A helpful link to the Git commit history since the last official release is also provided for further review.

Requirements

The following is a loose guideline. Other combinations of Go and operating systems for building and running tools from this repo may work, but have not been tested.

Building source code

  • Go
    • see this project's go.mod file for preferred version
    • this project tests against officially supported Go releases
      • the most recent stable release (aka, "stable")
      • the prior, but still supported release (aka, "oldstable")
  • GCC
    • if building with custom options (as the provided Makefile does)
  • mingw-w64
    • if building GUI app(s) for Windows
    • used to perform CGO-enabled builds of Fyne (GUI) applications
  • make
    • if using the provided Makefile
  • Fyle toolkit OS dependencies

Tip

Use make docker-release-build or podman-release-build Makefile recipes to generate/use build containers compatible with this project.

Running

The CLI apps are broadly compatible but have been tested against:

  • Microsoft Windows 10
  • Ubuntu 20.04

The GUI app(s) have been tested against:

  • Microsoft Windows 10
  • Microsoft Windows 11
  • Ubuntu 20.04
    • libgl1 package was needed

Note

The build image used by this project has an inherited dependency on the official upstream Go Docker image and shares that image's minimum glibc version requirement.

As of this writing, a glibc release of version 2.31 or newer is required to match the Debian 11 base image used by current Go Docker images. Ubuntu 20.04 has glibc 2.31 and meets this requirement. Older distro versions may not meet this requirement and will require building from source.

Important

When the upstream Go Docker image swaps out the minimum Debian base image OS (currently Debian 11) this may also mean dropping support for some Linux distros.

Installation

From source

Tip

Use make docker-release-build or podman-release-build Makefile recipes to use generate/use build containers compatible with this project.

  1. Download Go
  2. Install Go
  3. Clone the repo
    1. cd /tmp
    2. git clone https://github.com/atc0005/safelinks
    3. cd safelinks
  4. Install dependencies (optional)
    • for Ubuntu Linux
      • sudo apt-get install make gcc libgl1-mesa-dev xorg-dev
    • for CentOS Linux
      1. sudo yum install make gcc gcc libXcursor-devel libXrandr-devel mesa-libGL-devel libXi-devel libXinerama-devel libXxf86vm-devel
  5. Build
    • for the detected current operating system and architecture, explicitly using bundled dependencies in top-level vendor folder
      • most likely this is what you want (if building manually)
      • go build -mod=vendor ./cmd/usl/
      • go build -mod=vendor ./cmd/dsl/
      • go build -mod=vendor ./cmd/dslg/
      • go build -mod=vendor ./cmd/eslg/
    • manually, explicitly specifying target OS and architecture
      • GOOS=linux GOARCH=amd64 go build -mod=vendor ./cmd/usl/
        • substitute GOARCH=amd64 with the appropriate architecture if using different hardware (e.g., GOARCH=arm64 or GOARCH=386)
        • substitute GOOS=linux with the appropriate OS if using a different platform (e.g., GOOS=windows)
      • GOOS=linux GOARCH=amd64 go build -mod=vendor ./cmd/dsl/
      • GOOS=linux GOARCH=amd64 go build -mod=vendor ./cmd/dslg/
      • GOOS=linux GOARCH=amd64 go build -mod=vendor ./cmd/eslg/
    • using Makefile all recipe
      • make all
        • generates x86 and x64 binaries
    • using Makefile release-build recipe
      • make release-build
        • generates the same release assets as provided by this project's releases
  6. Locate generated binaries
    • if using Makefile
      • look in /tmp/safelinks/release_assets/usl/
      • look in /tmp/safelinks/release_assets/dsl/
      • look in /tmp/safelinks/release_assets/dslg/
      • look in /tmp/safelinks/release_assets/eslg/
    • if using go build
      • look in /tmp/safelinks/
  7. Copy the applicable binaries to whatever systems needs to run them so that they can be deployed

Note

Depending on which Makefile recipe you use the generated binary may be compressed and have an xz extension. If so, you should decompress the binary first before deploying it (e.g., xz -d usl-linux-amd64.xz).

Using release binaries

  1. Download the latest release binaries
  2. Decompress binaries
    • e.g., xz -d usl-linux-amd64.xz
    • 7-Zip also works well for this on Windows systems (e.g., for systems without Git for Windows or WSL)
  3. Copy the applicable binaries to whatever systems needs to run them so that they can be deployed

NOTE:

DEB and RPM packages are provided as an alternative to manually deploying binaries.

Deployment

  1. Place usl in a location where it can be easily accessed
  2. Place dsl in a location where it can be easily accessed
  3. Place dslg in a location where it can be easily accessed
  4. Place eslg in a location where it can be easily accessed

Note

The libgl1 package was needed on target Ubuntu systems for the dslg and eslg apps.

Configuration

Command-line arguments

  • Use the -h or --help flag to display current usage information.
  • Flags marked as required must be set via CLI flag.
  • Flags not marked as required are for settings where a useful default is already defined, but may be overridden if desired.

Note

🛠️ The dsl tool does not support CLI arguments but may do so in the future.

usl

Flags
Flag Required Default Repeat Possible Description
h, help No false No h, help Show Help text along with the list of supported flags.
version No false No version Whether to display application version and then immediately exit application.
v, verbose No false No v, verbose Display additional information about a given Safe Links URL.
u, url maybe No u, url Safe Links URL to decode
f, inputfile maybe No valid path to file Path to file containing Safe Links URLs to decode

NOTE: If an input url is not specified (e.g., via flag, positional argument or standard input) a prompt is provided to enter a Safe Links URL.

Positional Argument

A URL pattern is accepted as a single positional argument in place of the u or url flag. It is recommended that you quote the URL pattern to help prevent some of the characters from being interpreted as shell commands (e.g., & as an attempt to background a command).

Standard input (e.g., "piping")

One or more URL patterns can be provided by piping them to the usl tool.

An attempt is made to decode all input URLs (no early exit). Successful decoding results are emitted to stdout with decoding failures emitted to stderr. This allows for splitting success results and error output across different files (e.g., for later review).

Without arguments or flags

The usl tool can also be called without any input (e.g., flags, positional argument, standard input). In this scenario it will prompt you to insert/paste the URL pattern (quoted or otherwise).

dsl

Important

This tool is in early development and behavior is subject to change.

Flags

Note

🛠️ This feature is not implemented but may be added in the future.

Positional Argument

Note

🛠️ This feature is not implemented but may be added in the future.

Standard input (e.g., "piping")

Text with interspersed URLs (separated by whitespace) can be provided for decoding by piping it to the dsl tool. Output is sent to stdout.

Each matched Safe Links URL is replaced with a decoded version leaving surrounding text as-is.

Without arguments or flags

The dsl tool can also be called without any input. In this scenario it will prompt you to insert/paste content for decoding.

If no input is provided for a the listed amount of time the dsl tool will timeout and exit.

dslg

No command-line arguments are currently supported.

eslg

No command-line arguments are currently supported.

Examples

usl tool

Though probably not required for all terminals, we quote the Safe Links URL to prevent unintended interpretation of characters in the URL.

Using url positional argument

$ usl 'SafeLinksURLHere'
https://go.dev/dl/

Using url flag

$ usl --url 'SafeLinksURLHere'
https://go.dev/dl/

Using input prompt

In this example we just press enter so that we will be prompted for the input URL pattern.

$ usl
Enter URL: SafeLinksURLHere
https://go.dev/dl/

Using standard input (e.g., "piping")

$ cat file.with.links | usl
https://go.dev/dl/
http://example.com
http://example.net
$ echo 'SafeLinksURLHere' | usl
https://go.dev/dl/

Using filename flag

$ usl --filename file.with.links
https://go.dev/dl/
http://example.com
http://example.net

Verbose output

$ usl --verbose --url 'SafeLinksURLHere'

Expanded values from the given link:

  data      : PLACEHOLDER
  host      : nam99.safelinks.protection.outlook.com
  reserved  : 0
  sdata     : PLACEHOLDER
  url       : https://go.dev/dl/

dsl tool

Using url positional argument

Note

🛠️ This feature is not implemented but may be added in the future.

Using url flag

Note

🛠️ This feature is not implemented but may be added in the future.

Using input prompt

In this example we just press enter so that we will be prompted for the input URL pattern.

$ dsl
Enter single or multi-line input. Press Ctrl-C to stop (or wait 15s for timeout).

  - Feedback from this app is sent to stderr.
  - Decoding results are sent to stdout.
  - Tip: Redirect stdout to a file for multiple input lines.

Not shown is the copy/pasted content with Safe Links encoded URLs interspersed (e.g., right-click pasted into console).

Using standard input (e.g., "piping")

$ cat file.with.mixed.text.content | dsl
tacos are great https://go.dev/dl/ but so are cookies http://example.com
$ echo 'SafeLinksURLHere' | dsl
https://go.dev/dl/
$ dsl < file.with.mixed.text.content > decoded-output.txt
tacos are great https://go.dev/dl/ but so are cookies http://example.com

Using filename flag

Note

🛠️ This feature is not implemented but may be added in the future.

dslg tool

  1. Launch application
  2. Copy single URL or mix of URLs and text (e.g., copying an email) into the input field
  3. Press Decode button
  4. Press Copy to Clipboard button
  5. Paste decoded text where needed (e.g., a ticket)

Warning

The Copy to Clipboard action does not preserve any existing clipboard content; there is no undo for using this button

eslg tool

  1. Launch application
  2. Copy single unencoded URL or mix of unencoded URLs and text (e.g., copying an email) into the input field
  3. Press one of the desired "action" buttons
    • Encode All
    • Encode Randomly
    • QueryEscape All
    • QueryEscape Randomly
  4. Press Copy to Clipboard button
  5. Paste transformed text where needed (e.g., a new testdata file)

Warning

The Copy to Clipboard action does not preserve any existing clipboard content; there is no undo for using this button

License

See the LICENSE file for details.

References

General

Observed safelinks.protection.outlook.com subdomains

The following *.safelinks.protection.outlook.com subdomains have been found listed on various KB articles and forums:

Alternative decoders

The following alternative tools were encountered while researching valid Safe Links subdomains. These tools are listed here for informational purposes; no endorsement is implied.

Important

No guarantees are made regarding the functionality or privacy policies of the following online or local tools. YMMV.

Online decoders:

Local decoders:

safelinks's People

Contributors

atc0005 avatar dependabot[bot] avatar

Stargazers

 avatar

Watchers

 avatar

Forkers

uggyuggy

safelinks's Issues

Makefile: Refresh recipes to add "standard" set, new package-related options

Overview

Update/sync this project's Makefile against recent changes to the atc0005/check-cert project Makefile to provide the same functionality.

TODO

  • arch and OS-specific builds
  • release-build
    • used to generate just the binaries provided by a specific project (e.g., skip Windows (all), skip Linux x86)
  • links (and arch-specific variations)
    • used to generate download links for easier bulk retrieval of release assets
  • quick
    • go build without any custom settings applied
    • it is a quick build to test/prototype binaries without waiting for release-level optimizations to be applied
  • depsinstall
    • install build related dependencies
  • package generation
    • packages-stable
    • packages-dev

Add version metadata to Windows executables

Overview

Use github.com/tc-hib/go-winres and winres.json file to apply version details to Windows executables provided by this project.

Background

While this project currently does not provide Windows binaries as part of the normal release workflow, it may do so again in the future. Others may also wish to generate Windows binaries locally and could benefit from having version metadata embedded in the binaries for easier tracking and inventory management (e.g., SCCM deployments).

References

Add GUI app for decoding input text

Overview

Create GUI application to accept arbitrary input text and decode any Safe Links URLs found within leaving all other input text intact in the decoded output.

I've been reading up on the Fyne toolkit and plan to use it as the framework for the application.

The goal is to build Windows and Linux binaries for current OSes. Based on prototyping work Windows 10 and Ubuntu 20.04 are both fairly easily supported at this time.

Important

When the upstream Go Docker image swaps out the minimum Debian base image OS (currently Debian 11) this may also mean dropping support for some Linux distros.

References

Add tests using testdata input files

Proposed input files:

  • single non-Safe Links URL
  • single Safe Links URL
  • list of Safe Links URLs
  • mix of URLs
    • https:// with url query parameter (expected Safe Links URL pattern)
    • http:// with url query parameter (invalid Safe Links URL pattern)
    • http:// without url query parameter (invalid Safe Links URL pattern)
    • https:// without url query parameter (invalid Safe Links URL pattern)
  • email text with embedded Safe Links URLs with enclosing angle brackets
  • email text with embedded Safe Links URLs without enclosing angle brackets
  • email text with embedded URLs (no Safe Links URLs) without enclosing angle brackets
  • email text with embedded URLs (no Safe Links URLs) containing enclosing angle brackets
  • email text with mix of embedded URLs (Safe Links URLs and non-Safe Links URLs) with some URLs enclosed in angle brackets

Evaluate URLs enclosed within angle brackets as URLs

Overview

When evaluating release v0.5.0-alpha.5 with some test input emails I noticed that it failed to recognize URLs wrapped in angle brackets as URLs.

Some light digging later I learned that wrapping URLs in angle brackets is a common practice with Outlook to prevent splitting of URLs across lines. All current tools should be updated to consider URLs wrapped within angle brackets as URLs and then strip them before parsing.

References

Add initial prototype CLI tool

Small CLI tool to receive a Safe Links URL and decode to the terminal. If a verbose flag is provided emit a breakdown of the various query parameters and their values, otherwise emit minimal output.

-version does not output version number (but x.y.z)

Hi,

Thank's for sharing this tool.

Seems the -version does not output version number (but x.y.z)

Example with downloaded binary version v0.2.2

$ usl -version
usl x.y.z (https://github.com/atc0005/usl)
$

also tested with v0.2.3-alpha.1

Add support for processing multiple input lines

Hi,

Thank's for sharing this tool.

Adding below few ideas/feature requests. As they are quite "close" and/or linked to each other (I think), I list those in a single "issue".

It would be nice to have an option for "direct" results without extra informations lines and/or that even could be the default ouput and the current output with some kind of -interactive

$ usl 'https://emea01.safelinks.protection.outlook.com/?url=http%xxxxxxxxx'
example.net

Current output could also be

$ usl -interactive

Enter URL: https://emea01.safelinks.protection.outlook.com/?url=http%xxxxx
Original URL: http://example.net

It would be nice to be able to be able to bulk process multiple links at once in parallel

  • From piped data
$ cat file.with.links | usl
http://example.com
http://example.net
  • From input file
$ usl -inputfile file.with.links
http://example.com
http://example.net

Add support for decoding valid "Safe Links" URLs embedded within text

Overview / Scratch notes

Implementation:

  • allow piping text from an input file to an output file replacing all valid "Safe Links" (SL) URLs with their decoded equivalents
  • allow specifying a flag for an input file ...

The initial implementation would require that valid SL URLs are space separated from surrounding text otherwise would be left as-is (no errors raised).

This would allow for saving the text from an email to a plaintext file and creating a normalized copy of that file.

The initial implementation would probably also want to use different flags for an input file consisting entirely of URLs vs an input file containing mixed text of SL URLs and other content.

Additional thoughts (not fully thought out):

  • add a flag to warn if no matches are found (stderr output?)
  • add a second flag to trigger a specific exit code (without output) if no matches are found

After prototyping it may work out that multiple input file flags are not needed and auto-selection of behavior can be applied intuitively.

References

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.