voku / portable-ascii Goto Github PK
View Code? Open in Web Editor NEW🔡 Portable ASCII library - performance optimized (ascii) string functions for PHP.
License: MIT License
🔡 Portable ASCII library - performance optimized (ascii) string functions for PHP.
License: MIT License
Example: Is this correct?
"besøgende" -> "besoegende"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L589
Example: Is this correct?
"उदाहरण" -> "uThaHaRaNae"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L118
Example: Is this correct?
"Odwiedzający" -> "Odwiedzajacy"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L600
Example: Is this correct?
"ბავშვები" -> "bavshvebi"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L431
Example: Is this correct?
"müştəri" -> "musteri"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L1001
Example: Is this correct?
"univerzitní" -> "univerzitni"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L567
Seems like release v1.6.0 broke Laravel's build and subsequently all PR's that are sent in since are failing as well: https://github.com/laravel/framework/runs/4920401730?check_suite_focus=true#step:8:138
It seems most likely that this commit is the culprit: acaf868
Changing the behaviour here seems like a breaking change to me and should probably be done in a major release?
voku\helper\ASCII::to_ascii('пиздюк.txt');
Expected output: pizdyuk.txt
(pre 1.6.0)
Given output: pizdiuk.txt
(as of 1.6.0)
Think this is an easy fix by just rolling back the commit and tagging a new patch release.
/build
is missing :)
Thank you for maintaining this file
Example: Is this correct?
"päiväkoti" -> "paivakoti"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L424
MacOS filesystem replaces German (single character) umlaute with a two characters since Big Sur or only on M1 machines.
These are replaced with a simple a
instead of ae
. Expected behavior would be to replace these combined characters the same way as the single character umlaute are treated.
ä
, ö
or ü
- persist the filename (ENTER)ä/ö/ü
characterThe real-world scenario would be to sanitize an uploaded filename.
The local fix took minutes (added a str_replace()
before running the ASCII conversion) but no idea how much it will take to properly add it to the package.
In hope that GitHub doesn't replace the characters - here it is ä
- the codepoints are U+61 U+308
.
https://www.fileformat.info/info/unicode/char/61/index.htm
https://www.fileformat.info/info/unicode/char/308/index.htm
I'm not sure if this would be the expected behavior of the package or I want it to do something that's not in the scope of that package. We have a working solution right now - so it's in no way urgent.
Example: Is this correct?
"Ülikool" -> "Ulikool"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L653
Example: Is this correct?
"infanĝardeno" -> "infangxardeno"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L637
The recent release growed in size by ~45000 lines because of 5417140
We bundle the lib with our cms and it makes a big portion of our overall application size, see redaxo/redaxo#3662
Do you have any recommendations/suggestions on how we could save in filesize?
Maybe we can reduce the size of the data folder when we do some assumptions?
The current diff is pretty much only docs. But there is one super small change in composer.json.
Currently master contains a fix to a bug that makes my CI red. Could I please ask for a new patch release?
Example: Is this correct?
"vásárló" -> vasarlo"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L274
Example: Is this correct?
"университет" -> "universitet"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L548
Example: Is this correct?
"sveučilišt" -> "sveucilist"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L412
Some ASCII character mappings include a quote symbol:
portable-ascii/src/voku/helper/data/ascii_by_languages.php
Lines 1224 to 1225 in 0a05e9e
portable-ascii/src/voku/helper/data/ascii_by_languages.php
Lines 1228 to 1229 in 0a05e9e
Were these intended?
Example: Is this correct?
"zákazník" -> "zakaznik"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L1018
Example: Is this correct?
"trường học" -> "truong hoc"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L723
Example: Is this correct?
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L860
The Turkish character 'i̇' (capital 'I' with dot above) is not currently supported.
The most popular front-end framework for developing responsive, mobile first projects on the web.
Library home page: https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.1.3/js/bootstrap.min.js
Path to vulnerable library: /vendor/phpunit/php-code-coverage/src/Report/Html/Renderer/Template/js/bootstrap.min.js
Dependency Hierarchy:
Found in HEAD commit: 5f194d2b2d1490498ccef38efc358148119355c9
Found in base branch: master
In Bootstrap before 3.4.1 and 4.3.x before 4.3.1, XSS is possible in the tooltip or popover data-template attribute.
Publish Date: 2019-02-20
URL: CVE-2019-8331
Base Score Metrics:
Type: Upgrade version
Origin: twbs/bootstrap#28236
Release Date: 2019-02-20
Fix Resolution: bootstrap - 3.4.1,4.3.1;bootstrap-sass - 3.4.1,4.3.1
Step up your Open Source Security Game with WhiteSource here
JavaScript library for DOM operations
Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js
Path to vulnerable library: /vendor/phpunit/php-code-coverage/src/Report/Html/Renderer/Template/js/jquery.min.js
Dependency Hierarchy:
Found in HEAD commit: 5f194d2b2d1490498ccef38efc358148119355c9
Found in base branch: master
In jQuery versions greater than or equal to 1.2 and before 3.5.0, passing HTML from untrusted sources - even after sanitizing it - to one of jQuery's DOM manipulation methods (i.e. .html(), .append(), and others) may execute untrusted code. This problem is patched in jQuery 3.5.0.
Publish Date: 2020-04-29
URL: CVE-2020-11022
Base Score Metrics:
Type: Upgrade version
Origin: https://blog.jquery.com/2020/04/10/jquery-3-5-0-released/
Release Date: 2020-04-29
Fix Resolution: jQuery - 3.5.0
Step up your Open Source Security Game with WhiteSource here
Example: Is this correct?
"kundestøtte" -> "kundestoette"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L714
JavaScript library for DOM operations
Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js
Path to vulnerable library: /vendor/phpunit/php-code-coverage/src/Report/Html/Renderer/Template/js/jquery.min.js
Dependency Hierarchy:
Found in HEAD commit: 5f194d2b2d1490498ccef38efc358148119355c9
Found in base branch: master
jQuery before 3.4.0, as used in Drupal, Backdrop CMS, and other products, mishandles jQuery.extend(true, {}, ...) because of Object.prototype pollution. If an unsanitized source object contained an enumerable proto property, it could extend the native Object.prototype.
Publish Date: 2019-04-20
URL: CVE-2019-11358
Base Score Metrics:
Type: Upgrade version
Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11358
Release Date: 2019-04-20
Fix Resolution: 3.4.0
Step up your Open Source Security Game with WhiteSource here
JavaScript library for DOM operations
Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js
Path to vulnerable library: /vendor/phpunit/php-code-coverage/src/Report/Html/Renderer/Template/js/jquery.min.js
Dependency Hierarchy:
Found in HEAD commit: 5f194d2b2d1490498ccef38efc358148119355c9
Found in base branch: master
In jQuery versions greater than or equal to 1.0.3 and before 3.5.0, passing HTML containing elements from untrusted sources - even after sanitizing it - to one of jQuery's DOM manipulation methods (i.e. .html(), .append(), and others) may execute untrusted code. This problem is patched in jQuery 3.5.0.
Publish Date: 2020-04-29
URL: CVE-2020-11023
Base Score Metrics:
Type: Upgrade version
Release Date: 2020-04-29
Fix Resolution: jquery - 3.5.0;jquery-rails - 4.4.0
Step up your Open Source Security Game with WhiteSource here
Example: Is this correct?
"Ελληνικό αλφάβητο" -> "Ellhniko alfabhto"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L4
Example: Is this correct?
"IGÅR" -> "IGaR"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L185
Example: Is this correct?
"vaikų darželis" -> "vaiku darzelis"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L693
Example: Is this correct?
"діти" -> "diti"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L538
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
build/composer.json
voku/php-readme-helper ~0.6
composer.json
php >=7.0.0
phpunit/phpunit ~6.0 || ~7.0 || ~9.0
.github/workflows/ci.yml
actions/checkout v3
shivammathur/setup-php 2.17.1
actions/cache v2.1.7
codecov/codecov-action v2
actions/upload-artifact v3
Example: Is this correct?
"лысая гора" -> "lysaja gora"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L468
Example: Is this correct?
"français" -> "francais"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L1055
Example: Is this correct?
"دانشگاه" -> "danshgah"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L907
According to your description:
For example, German will map 'ä' to 'ae', while other languages will simply return e.g. 'a'.
this is wrong.
Example:
1 Apfel ( 1 Aple ) > nothing to convert
2 Äpfel (2 Apples) > you will convert Ä to A which is wrong because with an A only the 2 Apples become 1 Apple.
Maybe this sample is a bit hard to understand, but 1 Apfel = 1 Apple , 2 (or more) Äpfel are more Apples.
The only difference is the first letter, either an A or an Ä
Same will be with your sample Düsseldorf, converting it to en for example will translate Düsseldorf to Dusseldorf where it should be in all other languages than German Duesseldorf!
Dusseldorf may also exist as a town somewhere in Europe, if not Dussel means something very special in German (some stupid).
Translating German into other languages, especially for SEO-URLs is not that easy as it looks!
Please recheck and rework your framework, if it will handle such cases also correct, it could be useful.
Example: Is this correct?
"görüşmek" -> "gorusmek"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L193
Example: Is this correct?
"grădiniță" -> "gradinita"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L621
I've replaced (locally) Laravel's Str::ascii with a call to this package, however some of the results are not quite as expected. For example, the following test fails:
$this->assertSame('h H sht SHT a A y Y', Str::ascii('х Х щ Щ ъ Ъ ь Ь', 'bg'));
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'h H sht SHT a A y Y'
+'h H sht Sht a A '
Input: х Х щ Щ ъ Ъ ия йо
Output: h H sht Sht a A iya yo
Expected output: h H sht Sht a A ia yo
Example: Is this correct?
"တတျနိုငျသ" -> "ttyanongyath"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L294
Example: Is this correct?
"bērnudārzs" -> "bernudarzs"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L668
Example: Is this correct?
"универзитет" -> "univerzitet"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L926
In the process of updating several repos, will link pull request to this issue.
Example: Is this correct?
"пример" -> "primer"
Can a native speaker, please check the character-replacement, thanks.
https://github.com/voku/portable-ascii/blob/master/src/voku/helper/data/ascii_by_languages.php#L208
Hi there!
In case you missed it, I'm working on abandoning patchwork/utf8 in favor of symfony/symfony#33553 In this PR, I'm using transliterator_transliterate()
to implement UTF-8 to ASCII transliterations (see AbstractUnicodeString::ascii()
). But when the intl extension is not installed, I just skip the call and fallback to NFKC + maps + iconv.
It would be super cool if we could polyfill this function from intl (a subset of its behavior at least). This package is the closest I know. Would you consider porting it to https://github.com/symfony/polyfill/tree/master/src/Intl?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.