This work is related to the associated jambonz-webapp feature request.

The changes to the api-server will be two-fold:

1. The database and REST api will need to change to support the new properties being provisioned from the webapp
2. The process will need to expose a new websocket server endpoint, because we are going to use the listen verb, nested in a config, to stream the call audio to wss endpoint that the api-server exposes.

The first point is fairly self-explanatory.

the second deserves some more commentary. The api-server is an express app, and now it will be modified to mount a websocket server endpoint. The endpoint will be reached at the path /record, e.g. "https://jambonz.xyz/api/record". Once the websocket connection is established there will be a text frame sent with the call_sid and other metadata (as is done over all listen sockets). The api-server shall receive this frame and create a file with the given call_sid as the filename (with a filetype of .raw) and metadata that includes the account_sid and the sample rate. As the binary frames then come in with raw audio they should be uploaded to the file in the specified bucket.

in V0.7.3 client.calls.create fails with 'no available feature servers at this time'. This is caused by an incorrect lookup to redis for active feature servers rather than using the kubernetes feature-server service

As described in jambonz/jambonz-webapp#337

This will require a new table, google_custom_voices, which has a FK to the speech credentials table. We will need to capture:

• the google model
• the reported usage
• the language
• the voice name that the user will assign and use to refer to this voice.

Of course, apis and swagger will be updated as well

We need to add a "forgot password" link support to the webapp. This actually exists in the hosted (jambonz.xyz) system but not in the main one. So there is some code there to look at but we need some changes. Most of the changes need to be made here, in the api-server.

You will find some code in there that works with mailgun (e.g. requires mailgun api key). However, we cant force people to use mailgun they should be able to use any email provider. I think the best thing is that there is a new env var that gives an http endpoint we can post to that the customer will implement to send out mail. We'll probably need optional envs for http basic auth user and password as well. Then they can plug in their own email provider, and we can then add a "forgot password" link to the webapp that will generate the email via the http post.

This schema update should be part of the '8004' upgrades

Api to have token,accountid, vendor,label ,language, gender, voice as arguments to api, and in response it generates audio file

Currrently, on the feature/jwt-auth branch, the expiration of the jwt in the /login api is hardcoded to 1 hour. We should make this optionally configurable via an environment variable. Among other reasons, this will allow us to more easily test the jwt expired code path.

Hi @xquanluu, I'm testing recording with WAV format and it seems to be working all fine unless a DTMF code is sent. I've created a sample app at "https://public-apps.dev.bookline.io/hello" to reproduce the issue.

If you place a call with this app without pressing any DMTF code it's OK. If you place a call with this app and you press any digit at any time the recording is not created but I'm unable to see any error logs on the feature-server. Any hints on what we could troubleshoot here?

Currently we are querying the internal json api of jaeger using GET /api/v3/traces/${traceId}.

The problem is that while this works for the jaeger-all-in-one component, it does not work for the decomposed architecture where we call jaeger-query to retrieve the data. And the decomposed architecture (e.g splitting functionality into jaeger-agent, jaeger-collector, and jaeger-query) is required in order to save spans in cassandra persistent storage rather than memory.

Furthermore, the docs indicate the api we should be calling is GET /api/traces/${traceId}.

This api does work but it returns data in a different format:

• /api/v3/traces/${traceId} - sample data returned
• /api/traces/${traceId} - sample data returned

We need to change api-server to call /api/traces/${traceId} so that we can support both types of deployments (in-memory plus persistent storage). The change to api-server is simple but we will need to change the parsing / display logic in the webapp as well.

Currently, in a createCall POST request if an application_sid is provided, we retrieve the speech credentials for that application (ie as configured via the portal). However, if no application_sid is provided (instead urls for call_hook and call_status_hook are provided) then we are simply defaulting to google and en-US for speech. Instead, the createCall API should allow the ability to specify settings for tts and stt.

Add support for an optional metadata property to createCall. If provided, 'metadata' should be an object and this data should then be provided in the call_hook for the new call

{
    "application_sid": "0d27aee6-d1e6-4707-bdde-1cc20b50ca88",
    "from": "15083728399",
    "to": {
        "type": "phone",
        "number": "15082084809"
    },
   "metadata": {
      "customer": "acme",
      "referenceId: "deadbeef"
   }
}

The "Have your carriers whitelist our IPs" does not work for service provider users (or, presumably, account users):

I've been testing around with status events from my jambonz application but it seems I do not receive a 'failed' or 'no-answer' event for an outbound call if the the caller, who instantiated the dial, disconnects before the outbound destination answers. I see the following sequence of events when I'm testing:

1. Received inbound active call status event of type: trying for callId 6df26ca3-41c9-123c-2087-0a3949153b1f (call is instantiated)
2. Received inbound active call status event of type: in-progress for callId 6df26ca3-41c9-123c-2087-0a3949153b1f (call is answered & a dial is executed)
3. Received outbound active call status event of type: trying for callId 73158ed0-41c9-123c-a6b1-02a2c796aedb <= THIS IS WHERE THE CALLER HANGS UP
4. Received outbound active call status event of type: ringing for callId 73158ed0-41c9-123c-a6b1-02a2c796aedb
5. Received outbound active call status event of type: ringing for callId 73158ed0-41c9-123c-a6b1-02a2c796aedb
6. Received outbound active call status event of type: ringing for callId 73158ed0-41c9-123c-a6b1-02a2c796aedb
7. Received inbound active call status event of type: completed for callId 6df26ca3-41c9-123c-2087-0a3949153b1f

There should be a uniqueness constraint on sbc_addresses.ipv4, otherwise when sbc is restarted the same address is added twice. This leads to sign up errors as we try to add duplicate dns records

The createCall API in this case was initiated with an application_sid. It looks like the language setting was not retrieved and passed on the feature server:

{"level":20, "time": "2020-07-24T19:14:35.826Z","pid":25625,"hostname":"ip-172-31-33-250","body":{"application_sid":"9161d39c-acd2-4664-9f2a-89473ce9e698\\\\xxx","from":"15083084xxxx","to":{"type":"phone","number":"150830xxxx"},"timeout":15,"account_sid":"9351f46a-678c-43f5-b8a6-d4eb58xxx","call_hook":{"webhook_sid":"67dda29d-f430-44c7-92ad-b5aa681xxxxe","url":"http://app.jambonz.us/hi-bye","method":"POST","username":null,"password":null},"call_status_hook":{"webhook_sid":"cea2bf96-888f-4f81-889d-07f95823d76f","url":"http://app.jambonz.us/call-status","method":"POST","username":null,"password":null},"speech_synthesis_vendor":"google","speech_synthesis_voice":"en-US-Standard-C","speech_recognizer_vendor":"google","speech_recognizer_language":"en-US"},"msg":"got createCall request","v":1}

leading then to this error:

1|jambonz-feature-server  | {"level":50, "time": "2020-07-24T19:14:40.623Z","pid":25625,"hostname":"ip-172-31-33-250","generatedMessage":false,"name":"AssertionError [ERR_ASSERTION]","code":"ERR_ASSERTION","expected":true,"operator":"==","msg":"Error synthesizing text","stack":"AssertionError [ERR_ASSERTION]: synthAudio requires language when google is used\n    at synthAudio (/home/admin/apps/jambonz-feature-server/node_modules/@jambonz/realtimedb-helpers/lib/synth-audio.js:33:12)\n    at Promise.all.text.map (/home/admin/apps/jambonz-feature-server/lib/tasks/say.js:25:26)\n    at Array.map (<anonymous>)\n    at TaskSay.exec (/home/admin/apps/jambonz-feature-server/lib/tasks/say.js:24:53)\n    at process._tickCallback 

From twilio

On 10 October 2023 Twilio will be updating the media IPs and port ranges for SIP and Voice SDK calls in all 
regions to 168.86.128.0/18 and expanding the UDP port range to 10000-60000. Old IP and port ranges will no 
longer accept or send traffic after this date but will need to be kept open in your infrastructure until that time. You 
will need to update your network infrastructure to ensure that you have whitelisted the full IP and port ranges 
before 10 October 2023. Failure to do so will result in one-way audio and dropped calls.

In order to support retrieving a list of users at either the admin level (all users), service provider (all users for a given service provider and those for child accounts of a service provider) and account (all users for a given account), we need to add a new API.

To start with, this issue envisions simply retrieving all users in the database. It therefore shall be restricted to being called only by a user or api presenting a jwt or api key that has scope 'admin'.

We'll add complexity in future PRs, but this PR is simply to:

• update the swagger doc to describe the GET /Users (retrieveUsers) api
• implement the above (changes required mainly here), returning the attributes shown below for all users on the system
• add a simple test case exercising the above

user_sid
name
email
scope (calculated as admin, service_provider, or account based on existence of service_provider_sid or account_sid)
force_change
is_active

When clicking on 'create_db.sql' mentioned in the readme file returns 404 Not found.

Missing following fields on Update Call Body:
-child_call_hook

Missing following fields on Create Call Body:
-headers

Missing following fields on Target:
-vmail
-tenant
-overrideTo

Also missing the queues endpoint for querying queue counts

When using live call control to set a new call_hook, this works as a payload:

{
        call_hook: {
          url: '/tts-test'
        }
}

however, this does not work (and should)

 {
        call_hook: '/tts-test'
 }

When trying to create a call via the API the documentation allows different methods for creating calls. In Version 2 the webhook is explicitly provided in the payload in place of an application_sid. All attempts to use this seem to fail.

The only method that works is the method that sends in the application_sid, ie V1.

{ "application_sid" :"33c37050-d44b-495b-befc-XXXXXXXXX", "from": "3110xxxxxxx", "to": { "type": "phone", "number": "316xxxxx" } }

It would useful api that would return list of vendor supported by Jambonz

The branch feature/lcr has been opened to work on this feature. The api needs to support the needs of the webapp in creating LCR tables as described here.

The schema changes below have already been checked in on this branch:

• lcr represents an LCR table. It has a name and a default route. An LCR can be associated with either a single SP or account.
• lcr_routes represents digit match patterns in an LCR table. They have an explicit ordering (priority) and are evaluated in order from low to high priority value until a match is found.
• lcr_carrier_set_entry represents the carrier route to take when a digit pattern is matched. In the first version of this feature this will always resolve to a single carrier, but the schema is meant to support two other scenarios: route to a list of carriers in crankback fashion (try carrier 1, if fails try carrier 2), or distribution (send 80% of traffic to carrier 1, 20% to carrier 2).

LCR is an optional feature. A service provider may have an LCR table or not, ditto an account. An LCR table however is associated with only one service provider or account entity.

If I select AWS for speech synthesis, I get a list of voices. If I choose the default one it actually fails when I try to use it

e.g google asr return list all required json with techinal name its supported in jambonz

When setting the JAMBONZ_RECORD_WS_PASSWORD to a random 128-chars alphanumeric code, api-server started to throw this error and restart:

TypeError: Cannot read properties of null (reading '1')
at isValidWsKey (/opt/app/app.js:195:13)
at Server. (/opt/app/app.js:211:8)
at Server.emit (node:events:513:28)
at Server.emit (node:domain:489:12)
at onParserExecuteCommon (node:_http_server:903:14)
at onParserExecute (node:_http_server:797:3)
{"level":50,"time":1692699022976,"pid":1,"hostname":"api-server-6dc569dc54-m84zk","msg":"Uncaught exception error caught. Error was: Cannot read properties of null (reading '1')","v":1}

Looking a bit more into it, the issue is happening in this function:

const isValidWsKey = (hdr) => {
const username = process.env.JAMBONZ_RECORD_WS_USERNAME;
const password = process.env.JAMBONZ_RECORD_WS_PASSWORD;
const token = Buffer.from(${username}:${password}).toString('base64');
const arr = /^Basic (.*)$/.exec(hdr);
return arr[1] === token;
};

At first, I thought the issues was that arr was being null because the TypeError was clear pointing to the attempt of accessing index 1 of null. But by simply reducing the password to a 32-chars code, it started to work. So, my guess is that we have some sort of limitation on size here. And the error is not exactly where it appears to be looking into the logs.

We test Azure tts here using bent to do a simple GET; however bent does not support working through a proxy, and we have a need to support a configuration where all azure tts and stt requests go through an outbound proxy.

api-server already includes @jambonz/speech-utils, I think we should change the code to delegate the test to speech-utils and implement there using the microsoft-cognitiveservices-speech-sdk, which does support proxy.

Trying:

curl --location -g --request PUT 'http://[base url]/api/v1/Applications/[app sid]' \
--header 'Authorization: Bearer [api key]' \
--data '{
    "call_hook": {
        "url": "https:foo/bar"
    }
}'

What I expect to happen:

• set the call hook for my application to https://foo/bar

What actually happens:

{"msg":"Unknown column '{\n    \"call_hook\": {\n        \"url\": \"https:foo/bar\"\n    }\n}' in 'field list'"}

In the script to upgrade the DB schema in api-server file the upgrades that have to be performed are selected from a map by a version key which is a string (i.e. '8003') as per these these lines

However, a detailed look at the map shows that entries from version 8003 onwards are stored with the key as a number and not as a string

So the upgrades are not applied.

The entries should be stored with the key as a string instead of as a number.

Api to check if sip user is registered the argument to API token, accountid and sip username, API in response should give the registered status

This would be an optional feature to allow the initial jambonz application (i.e. the json array of verbs) to be stored in the mysql database. If supplied, the application will be retrieved from the database and immediately executed and the initial call_hook webhook will not be made.

This is mainly of use for highly-scalable applications that would benefit from fewer webhooks being invoked. For example, if you had a high volume application where you always start by playing a prompt/music or text, and expect a portion of callers to drop within the first few seconds; or if you had a relatively static application that doesn't change or get updated much, this feature might be useful.

Using swagger we get the following response for GET http://jsbc1.viva-eu.net:3000/v1/Accounts

{
  "msg": "Unknown column 'acc.device_calling_hook_sid' in 'on clause'"
}

Hi,
I have been trying to use the docker image ( using jambonz-infrastructure docker-compose) for the api-server , but login call fails when verifying hash in the argon2-ffi library.

{"level":50, "time": "2023-01-18T15:25:22.569Z","pid":1,"hostname":"e624e6d12fce","msg":"Database error","stack":"Error: Decoding failed\n at resultHandler (/opt/app/node_modules/argon2-ffi/lib/index.js:118:25)\n at Object. (/opt/app/node_modules/ffi-napi/lib/_foreign_function.js:115:9)","type":"Error","v":1}

I have tried different versions of the image, but to no avail, same error.

Node version in container image is v18.12.1

Using standard default admin user and password

References a voice name that was only available during alpha. This should be updated to "aura-asteria-en"

Related: jambonz/jambonz-webapp#304

In an AWS clustering environment, we made changes so that pm2 can start multiple instances of jambonz-feature-server. Each listens on an HTTP port, by default 3000, but if that port is claimed they will back off to the next one. So if we started 4 instances in pm2 they would be listening on ports 3000-3003.

However, when sending a REST api command like createCall to a feature server, the api-server always assumes the FS is listening on 3000, as we see here:

Thus all REST api commands would go to only one of the FS app instances. We should make it so that api-server is aware of the http listen ports of the various FS instances and round robins

The get calls api returns from the last 10 minutes, either in progress or finished. This is fine in normal operations, but when the system is under heavy load, the response body quickly becomes 1mb+ and response times creep up to 1000ms or more.

A filter option to only fetch calls in progress could help.

In some larger multi-tenant deployments it may be desirable to disallow API requests to set a default application at the carrier level. As an example, it is possible that many accounts will be using Twilio as a carrier, and in the first place we should not allow two of them to default route all calls from twilio. Currently, the first one that selects to default route twilio would "win" but as soon as a second set twilio as a default route calls would stop working for the first. It is not clear that we would want anyone in this scenario to be able to default route twilio; ie force everyone using twilio to enter a phone number and route based on that.

On the other hand, a customer entering their own SBC for a sip trunk we would want to be able to default route...

Register status api need to support array of user whose staus need to be checked

Hello @davehorton / @xquanluu,

I had a restart in the api-server last night, and when I inspected the logs in the old container I saw this error:

2023-08-30T00:52:00+02:00 {"level":30,"time":1693349520769,"pid":1,"hostname":"api-server-5b8d86759d-6j8gh","msg":"upgraded to websocket, url: /v1/record/aws_s3","v":1}
2023-08-30T00:52:00+02:00 {"level":30,"time":1693349520770,"pid":1,"hostname":"api-server-5b8d86759d-6j8gh","obj":{"sampleRate":8000,"mixType":"stereo","callSid":"f23fd4a2-60cd-4268-ac68-0a8ace3a0962","direction":"inbound",...}
2023-08-30T00:52:26+02:00 RangeError: start offset of Int16Array should be a multiple of 2
2023-08-30T00:52:26+02:00     at new Int16Array (<anonymous>)
2023-08-30T00:52:26+02:00     at PCMToMP3Encoder._transform (/opt/app/lib/record/encoder.js:18:21)
2023-08-30T00:52:26+02:00     at Transform._write (node:internal/streams/transform:175:8)
2023-08-30T00:52:26+02:00     at writeOrBuffer (node:internal/streams/writable:392:12)
2023-08-30T00:52:26+02:00     at _write (node:internal/streams/writable:333:10)
2023-08-30T00:52:26+02:00     at Writable.write (node:internal/streams/writable:337:10)
2023-08-30T00:52:26+02:00     at Duplex.ondata (node:internal/streams/readable:766:22)
2023-08-30T00:52:26+02:00     at Duplex.emit (node:events:513:28)
2023-08-30T00:52:26+02:00     at Duplex.emit (node:domain:489:12)
2023-08-30T00:52:26+02:00     at addChunk (node:internal/streams/readable:324:12)
2023-08-30T00:52:26+02:00 {"level":50,"time":1693349546310,"pid":1,"hostname":"api-server-5b8d86759d-6j8gh","msg":"Uncaught exception error caught. Error was: start offset of Int16Array should be a multiple of 2","v":1}

I need to check what call we were having at that moment to see if feature-server can say anything else. But at the moment the most important thing is to try to catch this error and avoid the crash of the service.

Refer: https://github.com/jambonz/jambonz-api-server/blob/main/lib/record/encoder.js#L16-L41

Thanks!

See jambonz/jambonz-webapp#320 for description

While recording a call with MP3 we get the following error:

RangeError: start offset of Int16Array should be a multiple of 2
at new Int16Array ()
at PCMToMP3Encoder._transform (/opt/app/lib/record/encoder.js:18:21)
at Transform._write (node:internal/streams/transform:175:8)
at writeOrBuffer (node:internal/streams/writable:392:12)
at _write (node:internal/streams/writable:333:10)
at Writable.write (node:internal/streams/writable:337:10)
at Duplex.ondata (node:internal/streams/readable:766:22)
at Duplex.emit (node:events:513:28)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9)

We are getting this error almost every time we record a call but we have been able to record some without this issue. Any hints on what info would be necessary to properly troubleshoot this?

In the webapp, if we open a recent call and that call doesn't have a recording in the bucket (due to a api-server update, the old calls don't have a recording yet), the api-server crashes with the following log (and a few seconds later I'm logged out of the webapp):

ApiError: No such object: my-bucket-recordings/2023/08/01/abcd5559-aaaa-1111-a12e-9a0f48fce600.mp3
    at new ApiError (/opt/app/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:80:15)
    at Util.parseHttpRespMessage (/opt/app/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:182:41)
    at Util.handleResp (/opt/app/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:156:76)
    at Duplexify.<anonymous> (/opt/app/node_modules/@google-cloud/storage/build/src/file.js:1072:38)
    at Duplexify.emit (node:events:513:28)
    at PassThrough.emit (node:events:513:28)
    at onResponse (/opt/app/node_modules/retry-request/index.js:234:19)
    at PassThrough.<anonymous> (/opt/app/node_modules/retry-request/index.js:165:11)
    at PassThrough.emit (node:events:525:35)
    at /opt/app/node_modules/teeny-request/build/src/index.js:191:27

With the google bucket vendor and using api-server 0.8.4-4

If the recording doesn't exist, perhaps don't show the waveform.

There doesn’t currently seem to be a documented REST endpoint to list and modify phone numbers at https://api.jambonz.org/. It used to be possible to call the undocumented /api/v1/PhoneNumbers with an account scoped API key. I understand there was possibly a security problem with this. At some point this has become /api/v1/ServiceProviders/:id/PhoneNumbers, but there is no /api/v1/Accounts/:id/PhoneNumbers. Service provider scoped works fine for me with an admin scoped API key on my private instances, but gives a 403 on the new jambonz.cloud instances as a user with an account scoped API key. I think this means that there is no way to access any PhoneNumbers API on public instances at present?

Why is there a discrepancy between the fields available at GetCall endpoint and from the RecentCalls endpoint?

Fields available in RecentCall but not in GetCall,

• attempted_at
• answered_at
• terminated_at
• termination_reason

Similarly there are some fields that are available in GetCall but not in RecentCall. My system fetches the call data from Jambonz periodically and we have to call both the APIs to get all the fields. It would be nice to have all the fields available in both

Something the webapp refresh has illuminated is that the Swagger docs for the API server have quite a bit of inaccuracies compared to actual DTO responses received on the client as well as at least one endpoint missing from the documentation entirely. It would be worth combing over the Swagger yaml and getting the documentation up to date. This issue may overlap with this one: #13 and we might be able to consolidate and close the older issue 🤔.

Todo for this issue:

• Create a "completed", comprehensive list of what is known to be missing or inaccurate in the Swagger docs

Comprehensive (incomplete) list of what is missing/inaccurate in Swagger docs:

• The API for Accounts/:account_sid/Applications is in use for the webapp but missing entirely from the docs
• The SmppGateways/SipGateways show inbound/outbound as boolean but they are actually binary: 1/0
• The SpeechCredential shows use_for_tts/use_for_stt as boolean but they are also binary: 1/0
• The RecentCalls shows the field as sip_call_id but it is actually sip_callid as returned from the API
• The :GET RecentCall and RecentCall pcap toggles have the same ID and both toggle open at the same time
• The paged responses for CDRs (RecentCalls, Alerts) have a different response schema where the docs claim a field called batch but instead the api returns a page_size field. Also the page and page_size fields are strings whereas the total field is a number and ideally they would all 3 be numbers 🤔
• ...

Per jambonz/jambonz-webapp#84 (comment) and jambonz/jambonz-webapp#84 (comment)

Currently, testing in combination with webapp main branch, I am allowed to add two speech service credentials for the same vendor.

How to test:

1. Add speech-service for "All accounts"
2. Add speech-service with the same vendor also for "All accounts"

Result: you will be allowed to create the second cred of the same vendor
Expected result: not allowed to create same vendor twice

On the Account level, it works as expected.