Comments (12)
Looks good 👍
I also like the idea of using the website meta data for choosing the "culture". Not sure if all servers provide info about the locale though...
from opendirectorydownloader.
Help is needed for a library which can automagically parse a lot of date formats.
from opendirectorydownloader.
This is still a very hard problem. Have tried to find a good datetime string parsing thing, but haven't found anything.
from opendirectorydownloader.
Could you share a few examples of dates you need to parse? :)
from opendirectorydownloader.
All of the formats in open directories.
I would've thought there would be at least one library which could do that. But couldn't really find one, tried multiple times.
Of course it can be written by hand, but preferring a library 😅
from opendirectorydownloader.
Maybe we could start a list of formats we encounter here in this thread? :)
So we can use the list to test different libraries or our custom code?
from opendirectorydownloader.
I'll start (you should be able to edit my comment to add new formats):
2019-12-08 01:19
2020-08-06 14:43:00
2020-08-6 14:43:00
26-Apr-2020 17:31
2019-Mar-17 03:43:25
2 months ago
(with tooltip2020-08-6 14:43:00
)March 28 2013 at 11:21
Went through the last month and top posts on /r/OpenDirectories...
from opendirectorydownloader.
Wow. Well done. Looks like these are all parseble quite good. The problem starts with the American dates 😅
Looks like I prefer Japanese dates 👍👍👍 Works very good on file systems.
Maybe later I'll create an option like the current --exact-file-size thing. Which also takes the date from the response, and probably also the ETag, and both can go into #20 😇
from opendirectorydownloader.
That sounds good! I'll keep looking for obscure date formats, but I believe these cover >90% of all submissions...
About parsing, I don't know about C#/.NET, but running all of the above dates throw JavaScript's new Date()
works for all except for 2 months ago
and March 28 2013 at 11:21
(although simply removing the at
did the trick). American dates (using /
as separator) were recognized correctly as well.
So if a plain built-in parser can handle this, it shouldn't be a huge problem :)
It's called japanese logic? Never knew that! I know it's the superior data format for everything and I try to use it wherever I can, but good to know there are actually people using this in day-to-day life :D
from opendirectorydownloader.
Seems like .NET should be able to parse most of these formats.
We should use a static "culture" format tough, so the app behaves consistently...
from opendirectorydownloader.
You are right. The CultureInfo (language) could maybe even provided/read by the website headers which makes it even better.
See fiddle for example of the parsing.
https://dotnetfiddle.net/HmZvn9
from opendirectorydownloader.
https://code-maze.com/csharp-convert-string-to-datetime/
from opendirectorydownloader.
Related Issues (20)
- Cloudflare HOT 11
- Support for http authentication HOT 5
- No URLs to save for AliceAndBooks.com HOT 6
- Crash when not specifying a protocol HOT 5
- No folder for LogArchives on systems with `/` as path seperator HOT 4
- SIGABRT on unknown cli args HOT 4
- Add version to JSON
- Add removal URL for uploaded files to JSON HOT 1
- Add mounting support HOT 5
- Actual OD Downloading support HOT 1
- Continous Downloading/Crawling depending on Timestamp HOT 1
- Copyparty `--force-js` support HOT 3
- Unsupported site HOT 1
- S3 support HOT 4
- 2.9.0.3 errors HOT 4
- Only single Chromium instance
- cant open in colab, giving error HOT 1
- JSON save error when using custom port HOT 1
- ways to bypass Cloudflare HOT 1
- How to Paste URLs into the program. HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opendirectorydownloader.