nezuo / lapis Goto Github PK

Failed to load profile for IraTenebrarum (#11211818): -- Promise.Error(ExecutionError) --

The Promise at:

ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.lapis.Collection:99 function load
ServerScriptService.TS.services.save-data-service:98 function onPlayerAdded
ServerScriptService.TS.services.save-data-service:83

...Rejected because it was chained to the following Promise, which encountered an error:

ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.lapis.Collection:92: attempt to index nil with 'GetUserIds'
ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.lapis.Collection:92
ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.Promise:172 function runExecutor
ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.Promise:181

Promise created at:

ReplicatedStorage.rbxts_include.node_modules.@rbxts.lapis.out.lapis.Collection:87 function load
ServerScriptService.TS.services.save-data-service:98 function onPlayerAdded
ServerScriptService.TS.services.save-data-service:83

here's where everything is being called:

private collection: Collection<DataStoreSaveType>;
	private documents: Map<Player, Document<DataStoreSaveType>> = new Map();
	private playerTroves: Map<Player, Trove> = new Map();

	constructor() {
		setConfig({
			saveAttempts: 5,
			loadAttempts: 20,
			loadRetryDelay: 1,
			showRetryWarnings: true,
			dataStoreService: DataStoreWrapper,
		});
		this.collection = createCollection(IS_STAGING ? "STAGING" : "PRODUCTION", {
			defaultData: serializeSave(defaultPlayerSave),
			validate: (data: DataStoreSaveType) => {
				print(data, deserializeSave(data));
				return true;
			},
		} as CollectionOptions<DataStoreSaveType>);
	}

	public onStart(): void {
		Players.PlayerAdded.Connect((player) => this.onPlayerAdded(player));
		Players.PlayerRemoving.Connect((player) => this.onPlayerRemoving(player));
	}

	private onPlayerAdded(player: Player): void {
		if (this.documents.has(player)) {
			player.Kick("Player already has a document loaded on this server");
			return;
		}
		this.collection
			.load(`Player${player.UserId}`)
			.then((document) => {
				if (!player.IsDescendantOf(game)) {
					document.close().catch((err) => warn(err));
					return;
				}
				//document.addUserId(player.UserId); // I thought this was the culprit
				const playerTrove = new Trove();
				store.setPlayerSave(player.UserId, deserializeSave(document.read()));
				playerTrove.add(
					store.subscribe(selectPlayerSave(player.UserId), (playerSave: PlayerSave | undefined) => {
						if (!playerSave) return;
						document.write(serializeSave(playerSave));
					}),
				);
				playerTrove.add(() => {
					if (this.documents.has(player)) {
						this.documents.get(player)!.close();
						this.documents.delete(player);
					}
				});
				this.documents.set(player, document);
				this.playerTroves.set(player, playerTrove);
			})
			.catch((err) => {
				warn(`Failed to load profile for ${player.Name} (#${player.UserId}): ${err}`);
				player.Kick("Failed to load save file, please check back later.");
			});
	}

Lapis is currently dependent on game:BindToClose during module import and this event seems to not work in projects with Team Create enabled.

Right now, the throttle queue only processes one request at a time. This means if you loaded a document and it kept retrying because it was session locked that would stop all other load/save/close requests. This is obviously not ideal.

Fixing this would mean that the order of UpdateAsync calls for different keys is not guaranteed.

If a document loads after game:BindToClose it won't be closed unless the user is closes it sometime after manually (for example PlayerRemoving).

The question is, how should it stop the loading?

• Resolve the load promise with nil instead of a document?
• Reject the load promise?
• Infinitely yield the load promise?

ProfileService just returns nil from ProfileStore:LoadProfileAsync, however, I think infinitely yielding is the best option. This is because the other 2 solutions could cause user code to error. This shouldn't result in an error because it's not a bug. It would be unfortunate if a user saw an error and was worried about it.

Lapis session locks expire after 30 minutes. This is not documented yet even though it should be.

It should be possible to modify the data of a document before it loads. For example, you could implement reconciliation (think ProfileService:Reconcile) or decompression of the document.

Some open questions:

• Should this callback run before or after migration? Should there be a callback for both?
• Should this callback be global or per collection?

I'd also like to add a reconciliation example to the docs after this feature is added.

Example implementation:

UpdateAsync(function(oldData)
    if oldData.lockId == document.lockId then
        oldData.lockId = nil
        return oldData
    end

    return nil
end)

As long as the current lock is still the same as the documents, it should be safe to remove it.

Currently, it's not possible to make final changes to a document when Document:close is called by game:BindToClose.

To solve this, I propose adding Document:beforeClose(callback: (data) -> ()). The document would call all the registered callbacks inside of Document:close before it saves the data.

Lapis keeps it's own throttle queue so every request eventually gets processed. The downside is that the throttle queue can infinitely grow, causing documents to not be saved for a long time (or even before the server closes). Lapis should warn if the throttle becomes large enough that the user is doing something wrong.

Overview

Session locking cannot (currently) be disabled, and for good reason; it seems that lapis is designed with session locking in mind. Although, it would be great if it wasn't.

Use case

I'm working on a game that has a system similar to "clans" or "guilds". These have mutable data elements that can be changed across multiple servers, eg;

• Members joining and leaving the guild
• Guild details updating
• Some other game specific features

As these elements can be mutated across multiple servers- they have to be sync'd every N minutes, and updates have to be in the form of UpdateAsync with a provided callback that handles reconciliation between guilds. This reconciliation would be a great place to take advantage of lapis' Document abstraction.

Closing thoughts

This might be outside the scope of lapis' design philosophy, but I thought I'd suggest it just in case it's something you're interested in. I'd really like to take advantage of lapis in my current project- instead of needing to write my own wrapper around data stores; although, this may also just be a space that needs filling by a separate project/service (multi-server data store access).

When a profile fails to load (e.g. validation fails) the session lock is left on the data and is not released.

Additionally, when attempting to load at a later time, the session lock is also not overridden, meaning crashed or hanging servers will permanently prevent data from being loaded because the session lock never expires and is never overridden.

The default saveAttempts setting is 5. Since there is no save retry delay, this means closing a document will only take the amount of time those 5 UpdateAsync calls take. That's a lot less then the 20 seconds loading is retried for.

It would be useful to have a way to read a player's data while they're not in the game. For example, if there's a leaderboard system where it shows the top 3 players on podiums and you want to show things they own like skins, guns etc. The read-only feature would allow you to load their data and hold it temporarily to extract the value(s) you need.

I believe ProfileService has a similar feature.

Thanks! 👍

Is there support for DataStore.SetAsync's userIds parameter (or UpdateAsync's version of it) to help with GDPR removal requests? If not, is it planned?

At the moment, Lapis automatically deep freezes your document's data. This makes it impossible to work with mutable data. There are two options here:

1. Mutable by default
2. Immutable by default

As of right now, I think we should go with the second as it's what current users of Lapis expect to happen.

Having an option to disable immutability can also be useful for users who are already deep freezing their documents.

Should this option be global or per collection?

There should be an example of how to properly handle the MarketPlace:ProcessReceipt callback to make sure data saves before granting the purchase.