Comments (8)
Hi Tim,
we can't import TextEncoder from 'util' because 'util' is only available in node. Later node versions (not sure which) make TextEncoder available in the global scope, like web browsers do.
There is a workaround:
import {TextEncoder} from "util";
import {BinaryWriteOptions, BinaryWriter} from "@protobuf-ts/runtime";
const myWriteOptions: BinaryWriteOptions = {
// set a custom writer factory that passes the node
// text encoder to the writer
writerFactory: () => new BinaryWriter(new TextEncoder()),
writeUnknownFields: true
};
// assuming you have a message type "Test" and an instance of it "t1"
let bytes = Test.toBinary(t1, myWriteOptions);
Let me know if this works for you.
Could you check if globalThis.TextEncoder
is defined in your environment? (JS or TS, should not matter).
If so, new TextEncoder() could be replaced with new globalThis.TextEncoder()
, and the workaround would not be necessary.
from protobuf-ts.
Ah that makes sense. Unfortunately globalThis
is defined but globalThis.TextEncoder
is not. Looking at the node docs, they claim that it was added at least at some point in v11.X. I was on v12.16.0, and updated to the latest v12.19.0, but I still didn't see it. (I can't really go up to 14 for instance due to my application).
Thank you for the help, though.
from protobuf-ts.
Same problem in Node 14.x
from protobuf-ts.
Same problem in Node 14.x
This project is running continuous integration on Node 14.5, and there are many tests of the runtime library that will hit TextEncoder, and there are also conformance tests that run on multiple variations of the generated code, and we haven't seen this error yet.
I've just tested on Node v14.19.0 and cannot reproduce either.
Can you give more details about your environment? The exact Node version, code generation parameters and the module system being used?
from protobuf-ts.
Thank you for the quick answer @timostamm
my environment is:
NodeJS: v14.18.3
Typescript: v4.4.3
Jest: v26.6.1
The error occurs inside my own unit test using Jest. To be sure, I printed process.version
variable and got '14.18.3'.
We have experienced the lack of TextEncoder in the pass in nodeJS and electron. To solve that, we have an util function like this:
function encodeUtf8String(input?: string): Uint8Array {
if (isNodeJS()) {
return new Uint8Array(input ? [] : Buffer.from(input, 'utf-8').buffer);
} else if (isBrowser()) {
return new TextEncoder().encode(input);
} else {
throw new Error('Unknown environment. Encoding is not possible.');
}
}
to avoid having to deal with this in multiple places.
I created a custom TextEncoderLike using my function, and it's working fine.
It would be nice if something similar could be the default behavior in the BinaryWriter.
from protobuf-ts.
I can't reproduce with node v14.18.3 either. I suspect this is related to transpilation, or how node is run.
I have another project here that also uses TextEncoder where I have never seen this issue. The sources are compiled with typescript v4.5.5 to CommonJS, then run with jest v27.5.1.
tsconfig.json
{
"include": [
"src/**/*.ts"
],
"compilerOptions": {
"target": "es2020",
"lib": [
"ES2016",
"ES2020.BigInt"
],
"strict": true,
"importsNotUsedAsValues": "error",
"noImplicitAny": true,
"strictNullChecks": true,
"strictFunctionTypes": true,
"strictBindCallApply": true,
"strictPropertyInitialization": true,
"noImplicitThis": true,
"useUnknownInCatchVariables": true,
"noUnusedLocals": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"noImplicitOverride": true,
// We need node's module resolution, so we do not have to skip lib checks
"moduleResolution": "Node",
"skipLibCheck": false,
// We need correct line numbers in jest test failures
"sourceMap": true
}
}
jest.config.js
/*
* For a detailed explanation regarding each configuration property and type check, visit:
* https://jestjs.io/docs/configuration
*/
/** @type {import('@jest/types').Config.InitialOptions} */
const config = {
// Indicates which provider should be used to instrument code for coverage
coverageProvider: "v8",
// The root directory that Jest should scan for tests and modules within
rootDir: "dist/esm",
};
export default config;
Jest is run with:
npx tsc --project tsconfig.json --module commonjs --outDir ./dist/cjs
npx jest
How do you run Jest? Using Node's Buffer is an option, but I'm very wary of that because it needs bypassing all linters and type checks. So I'd like to understand what's going on first.
from protobuf-ts.
Let me roll back my TextEncoderLike and perform some test to understand what could be the cause that TextEncoder is not available.
I'm executing Jest directly by invoking 'jest' with a jest.config.json a little more elaborated than yours. Also, the tsconfig.json is different.
from protobuf-ts.
FWIW, I got the same error. I solved the problem by upgrading to the latest Jest and ts-jest versions.
from protobuf-ts.
Related Issues (20)
- unused parameters in the empty.ts are causing TypeScript check errors HOT 6
- Global `Object` is not referenced when there is a message named Object. HOT 2
- Proto2 required fields are not serialized when they are the default value HOT 3
- Is this compiling error could be turned off by Typescript or Option in protobuf-ts HOT 2
- 2.9.2 not working with 'exactOptionalPropertyTypes: true' HOT 1
- Release 2.9.3 breaks serialization for me HOT 6
- Generated message classes cannot be subclassed because of missing 'export' HOT 1
- Error decoding from binary: "cant skip wire type 4" HOT 1
- How to debug INVALID_ARGUMENT? HOT 1
- Missing headers on RPCError HOT 8
- rpcError cannot support Chinese
- [Request] Flag to handle google wrapper types HOT 1
- Vite cannot found module HOT 2
- Invalid typescript in descriptor.ts HOT 2
- Can't retry request after awaiting response HOT 2
- Typesafe API to retrieve enums from strings HOT 4
- `output_javascript` no longer emits `.d.ts` files HOT 6
- Request: Support for moduleResolution "nodenext" and "node16" HOT 2
- Fails to download v27.0-rc1 (which was just released) HOT 7
- invalid url for latest v27.0-rc1 release HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protobuf-ts.