Git Product home page Git Product logo

urlexpand's Introduction

URLEXPAND

Crates.io Documentation Build Status GitHub license

Expand / Unshorten an exhaustive list of Shortened URL's.

Example Usage

  • async
urlexpand = "0.2.3"

and then

use std::time::Duration;
use urlexpand::unshorten;

fn main() {
    let url = "https://bit.ly/3alqLKi";
    assert!(unshorten(url, Some(Duration::from_secs(10))).await.is_ok());   // with timeout
    assert!(unshorten(url, None).await.is_ok());    // without timeout
}
  • blocking
urlexpand = { version = "0.2.0", features = ["blocking"] }

and then

use std::time::Duration;
use urlexpand::unshorten;

fn main() {
    let url = "https://bit.ly/3alqLKi";
    assert!(unshorten_blocking(url, Some(Duration::from_secs(10))).is_ok());   // with timeout
    assert!(unshorten_blocking(url, None).is_ok());    // without timeout
}

Running the example

cargo run --example unshorten https://bit.ly/3alqLKi

Current list of URL Shortening services supported

  • adf.ly - Adfly
  • adfoc.us - AdFocus
  • amzn.to - Amazon
  • atominik.com - part of Adf.ly
  • ay.gy - part of Adf.ly
  • b.link - BLINK
  • bhpho.to - BH Photo & Video
  • bit.ly - Bitly Url Shortner
  • bit.do - Bitdo
  • bn.gy - BNGY
  • branch.io - Branch.io
  • buff.ly - Buffer URL Shortner
  • ceesty.com - part of sh.st
  • chollo.to - Chollo Ecommerce
  • cli.re - Capsulink
  • cli.fm - Capsulink
  • cutt.ly - Cuttly
  • cutt.us - Cutt us
  • db.tt - Dropbox
  • f.ls - Free Link Shortener
  • fa.by - part of rebrand.ly
  • fb.me - Facebook
  • flip.it - Flipboard
  • fumacrom.com - part of Adf.ly
  • git.io - Github
  • geni.us - Genius Link
  • goo.gl - Google Service has now stopped their url shortening service
  • gns.io - part of trim (tr.im)
  • hmm.rs - HMM.RS
  • ht.ly - part of Hootsuite
  • hyperurl.co - SmartUrl.It
  • ity.im - ity.im (it'-ee-i-am)
  • intamema.com part of Adf.ly
  • is.gd - IS GD
  • j.gs - part of Adf.ly
  • j.mp - part of Bitly
  • kutt.it - Kutt
  • ldn.im - part of trim (tr.im)
  • linklyhq.com - Linkly HQ
  • microify.com - part of Adf.ly
  • mzl.la - Mozilla Org
  • nmc.sg - instra corporation pty
  • nowlinks.net - Now Links
  • ow.ly - part of Hootsuite
  • prf.hn - Partnerize
  • plu.sh - Plush
  • q.gs - part of Adfly
  • qr.ae - Quora
  • qr.net - QR Code URL shortner & generator
  • rebrand.ly - Rebrandly
  • rb.gy - RBGY Free URL Shortner
  • rlu.ru - RLU.RU
  • rotf.lol - part of tinyurl.com
  • s.coop - SCOOP
  • s.id - SID (home.s.id)
  • sh.st - shorte.st
  • soo.gd - Soo.Gd
  • shortcm.xyz - part of SHORT.IO
  • short.gy - SHORT.IO Service
  • shortcm.xyz - ShortCm
  • shorturl.at - ShortURL At
  • smu.sg - Singapore Management University
  • smq.tc - part of bit.ly
  • snip.ly - Sniply.io
  • snipr.com
  • snipurl.com
  • snurl.com
  • split.to - Linksplit
  • surl.li - Hyperhost (Secom.com.ua)
  • t.co - Twitter
  • t.ly - T.LY Link Shortener
  • t2m.io - T2M aka "Text to Marketing" (t2mio.com)
  • tiny.cc - TinyCC
  • tiny.pl - TinyPL
  • tinyium.com - part of Adf.ly
  • tinyurl.com - TinyURL
  • tiny.one - part of tinyurl.com
  • tny.im - TNY.IM
  • tny.sh - Linksplit
  • tr.im - trim (tr.im) by RedLotus
  • trib.al - Tribal links shortner
  • u.to - U TO
  • v.gd - V GD
  • virg.in - Virgin
  • vzturl.com - Vzt URL
  • waa.ai - Akari Link Shortner
  • washex.am - part of bit.ly
  • x.co - GoDaddy URL Shortner (currently shutdown)
  • y2u.be - YouTube URL Shortner by Firewrench inc.
  • yourwish.es - Your Wishes
  • zpr.io - Zapier

Contribution

Please feel free to contribute by making pull requests or even bug fixes.
Thanks in advance.

License: MIT

urlexpand's People

Contributors

marirs avatar rnbguy avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

rnbguy orin220444 steadroy alexander-bui

urlexpand's Issues

REQUEST ERROR? AFTER 1000 LINKS or maybe because a link was double cloaked (shortened twice)

BRO, I LOVE YOUR CODE, listen, i'm not a programmer, but exactly 8 hours ago I installed RUST for the first time, fast forward, exactly 30 minutes ago I was already using your code expanding 30,000 links, thanks for your work

HOWEVER, I would like to discuss if there are going to be limitation to how many request we can pull in one execute? There was an instance where it went error after the 1,030th request, perhaps because the last request had a link cloaked twice from t.co to bit.ly, but there was a final destination yet to be expanded, do you think this was a limitation to the program or do you think it was the former?

ANYWAY, I will post below my code for unshortening 30,000 links, I'm really new to this GitHub thing and I can't find a way to contact you so here I am, cheers! Hope u reply! See ya

use std::io::Read;
use std::fs::File;
use std::io::Write;
use urlexpand;

#[tokio::main]
async fn main() {
    let mut file = std::fs::File::open("data.csv").unwrap();
    let mut save_file = File::create("data_new.csv").unwrap();
    let mut contents = String::new();
    file.read_to_string(&mut contents).unwrap();
    println!("{}", contents);

    for url in contents.lines() {
        let x = urlexpand::is_shortened(&url);
        if x {
            match urlexpand::unshorten(&url, None).await {
                Ok(u) => { writeln!(&mut save_file, "{}", u).unwrap(); },
                Err(e) => println!("{}", e),
            }
        } else {
            println!("{} not a short url", url)
        }
    }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.