Git Product home page Git Product logo

riiablo's Introduction

Riiablo

Join us on Discord!

This is my attempt at rebuilding Diablo II from scratch. There is still a long way to go, but a lot of the core is there. The game itself uses 100% original Diablo II assets which are not (and will never be) provided, i.e., you should already own a copy of the game to play this. I am going to do my best to remain faithful to the original game, however I am planning on changing and adding additional features as I think they become necessary (e.g., auto gold pickup). I will write articles in the future explaining how I was able to accomplish certain things, and how some core systems work.

Unlike the original game, this version supports just about any reasonable aspect ratio (4:3, 16:9, 18:9, 21:9), however by design I'm locking the game to 480px in height because the original game assets are all geared for that (i.e., the width is dynamic). The expansion did introduce 800x600 support, and I may end up supporting for the desktop version in the future (specifically for some multiplayer lobby stuff), but it's not necessary for the core functionality. The mobile version currently uses 360px height to make selecting UI elements easier, however since much of the UI panels are 480px, in-game is locked at 480px, but I will change this in the future or at least provide scaling on a per-component basis (text). This does run on Android, and I have been using a Galaxy Note 5 as the min spec when testing, but older phones may work as well, especially after optimizations have been made. I can already play Diablo II on PC, my goal is to be able to sit back and play it casually with my friends while also supporting cross-platform play. This game supports local play that can then be taken online (similar to Open Battle.net), with a more secure option being far beyond that.

NOTE: This is not playable yet, but the game runs and you can load save files, walk around a bit and look at your characters. Game saves are not modified yet, and 1.13c+ saves are supported (support for some other versions may be added in the future, but it isn't a priority, and I expect most people to create new characters anyways). I do not plan on, or want to make this game compatible with playing with users using the original game client.

SP Test

Features

  • Written using Java + LibGDX + OpenGL + Flatbuffers + Netty
  • Runs on PC, Android and eventually more (IOS, Linux, etc.)
  • Cross-platform multiplayer
  • Dedicated servers, TCP/IP (listen servers), and single player
  • Full console, including CVAR support and custom key bindings
  • Controller support
  • Platform-specific features (Android touch, PC mouse, etc)
  • Scalable UI

Screenshots

In-Game Create Character Paladin Android MP Test

Setup

The Android SDK is required to build the android subproject project.

Two environment variables can be used to automatically set the D2 installation and save directories if they are not automatically detected, and you do not want to use command-line arguments every time you launch the game.

D2_HOME=/Diablo2
D2_SAVE=/Diablo2/Save

Otherwise, using the --help command-line argument will show a list of all available options -- including manually specifying your D2 installation.

Building

git clone https://github.com/collinsmith/riiablo.git
cd riiablo
gradlew desktop:run

Windows

Typical D2 installations should be automatically detected and configured (including existing saved games). If a D2 installation cannot be detected (or you would like to do something like change which saved games to use), see the above instructions on using command-line arguments.

Linux / MacOS

This project was developed using the native Win32 MPQ files, so you will need to copy them from your Windows installation (whether that be Wine or just a copy of the files). Detection of the MPQs may be spotty and require manually specifying them via command-line arguments or environment variables (see above).

Android

Debug APKs can be created with gradlew android:assembleDebug, however configuring the app on a device is a bit of a pain at this time unless you can manually copy the resources from your Win32 installation onto your device in the app data directory. This will require having logcat running so that you can see any errors it spits out about where it's looking for the files. This process will be made easier in the future.

IntelliJ

  • Importing into Intellij/Android Studio
  • Default run configurations are provided within .idea/runConfigurations. The default resolution is 854x480, other configurations are provided to ensure a wide range of support --windowed arg can be used to start in windowed mode, while F12 can be used in-game to disable the debug UI.
  • Environment variables D2_HOME and D2_SAVE can be used if you are a pluggy user.

Tools

Aside from the main tools below, other tools are contained within :tools. To view all available tools, use gradlew :tools:projects -q. --help is your friend.

MPQ Viewer

Allows you to look at the game's assets and debug issues with specific files. Not all file types are supported at this time, but those should hopefully come in the future. This is a tool used in development and is not a replacement for tools like DR Test, but does provide the same functionality.

Screenshot MPQ Viewer

Map Viewer

Tests the map building algorithm/renderer. This is very basic at the moment.

Screenshot Map Builder

Developed Using:

IntelliJ IDEA Logo

riiablo's People

Contributors

collinsmith avatar jansimek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

riiablo's Issues

patch_d2.mpq error reading data\global\sfx\item\gem.wav

patch_d2.mpq version of data\global\sfx\item\gem.wav is failing to load due to bad data. Flags are marked as IMPLODE|EXISTS, but this doesn't seem to be the case (it looks like it's encrypted or compressed still like in d2sfx.mpq). d2exp.mpq also contains data\global\sfx\item\gem.wav, but it's a different sound. Since the sound in d2sfx.mpq sounds the same as patch_d2.mpq, I'm going to manually override it and circumvent d2exp.mpq as a workaround.

I'm not sure if this is an error with the MPQ or an error with my decoder, however my inclination is that this is an error with the MPQ since I've yet to see any other files produce such an error (although other MPQ readers are able to extract it). If any more pop up, I'll have this issue to reference.

[MPQ] Reading data\global\sfx\item\gem.wav...
[MPQInputStream] Populating sector offsets table
[MPQInputStream] sector offsets = [36, 4132, 8032, 11939, 15789, 19373, 22970, 26503, 28401]
[MPQInputStream] Reading sector 1 / 8 IMPLODE|EXISTS
[MPQInputStream] Exploding sector...
CSize=4096
FSize=4096
sector(first 32 bytes)=[52, 49, 46, 46, 82, 7A, 00, 00, 57, 41, 56, 45, 66, 6D, 74, 20, 10, 00, 00, 00, 01, 00, 01, 00, 22, 56, 00, 00, 44, AC, 00, 00]
Exception in thread "LWJGL Application" com.badlogic.gdx.utils.GdxRuntimeException: Unable to read file: data\global\sfx\item\gem.wav
	at com.riiablo.mpq.MPQInputStream.readBytes(MPQInputStream.java:398)
	at com.riiablo.mpq.MPQ.readBytes(MPQ.java:106)
	at com.riiablo.mpq.MPQ.readBytes(MPQ.java:90)
	at com.riiablo.mpq.MPQFileHandle.readBytes(MPQFileHandle.java:28)
	at gdx.diablo.tester.rempq.REMPQTest.create(REMPQTest.java:68)
	at com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(LwjglApplication.java:149)
	at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:126)
Caused by: java.lang.IllegalArgumentException: PK_ERR_BAD_DATA: Invalid LitSize: 82
	at com.riiablo.mpq.util.Exploder.pkexplode(Exploder.java:167)
	at com.riiablo.mpq.MPQInputStream.readBytes(MPQInputStream.java:369)
	... 6 more

FYI: Output from decoding d2sfx

[MPQ] Reading data\global\sfx\item\gem.wav...
[MPQInputStream] Populating sector offsets table
[MPQInputStream] Decrypting sector offsets table
[MPQInputStream] sector offsets = [36, 4132, 5631, 7128, 8622, 10112, 11600, 13091, 14972]
[MPQInputStream] Reading sector 1 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 2 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 3 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 4 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 5 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 6 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 7 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 4096 bytes
[MPQInputStream] Reading sector 8 / 8 COMPRESSED|ENCRYPTED|FIX_KEY|EXISTS
[MPQInputStream] Decrypting sector...
[MPQInputStream] Decompressing sector...
[MPQInputStream] Decompressed 2698 bytes

CharacterPanel resistances coloring

This table works a bit differently then the others. In this case

x<0 red
x>0 white
x==max gold
modified by non-items blue

Additionally, max(fire|cold|light|poison|magic)resist are not the maxstat for the resist stats, so these max values need to be hard coded.

Certain special unique items don't use correct image/sounds

Some items, e.g., Annihilus and Hellfire Torch, use a base item type that does not match their appearance/sound set. These appear to be hardcoded in the client. Changing the item base to the matching one (e.g., set Annihilus to mss base from cm1) is not sufficient, as this changes other properties of the items which are normally retrieved via item.base. fields. I think the proper fix would be to remove references to affected fields and implement methods in Item itself. Other items that typically have this kind of behavior only change the image and have separate entries (e.g., Viper Amulet, Staff of Kings, Hellforge Hammer, etc). It's possible this is the way it is because these items are uniquely identifiable, and thus when unidentified need to appear for all intents and purposes as their normal counterparts, sounds and all.

Affected fields: flippyfile, invfile, uniqueinvfile, dropsound, dropsfxframe, usesound

unique item custom images/sounds

Item should be modified so that if the item is identified and unique and pictureId > 0 then use the unique file and image from UniqueItems.txt. This will clean up some of my hacky logic to fix Annihilus and Hellfire Torch. Additionally, UniqueItems.txt contains the appropriate drop sounds that can be used.

Den of Evil naming

It appears that the Den of Evil is either named the "Den of Evil" or "Cave Level 1" depending on whether or not Q1 is active. The quest version also has some special properties, such as unique environment ambient color (red). AKA, the LevelName, LevelWarp and EntryFile don't match up and must be set with code. It seems that this only a naming/environment change though, as the monsters are the same and it uses the same level generation (also 1 level not 2 like the actual "Cave" in the Cold Plains). This will have to be looked at more when quests are added, and I'm betting this is a global event, i.e., the naming is changed when the level is generated and doesn't display differently depending on player quest status.

Project name might be legally problematic

IANAL, but it might be legally problematic (and get this interesting project shut down) unless you change name to something that sounds a bit less exactly like Blizzard's IP (intellectual property).

Also (kind of relevant when it comes to naming) have you heard of OpenD2 (https://github.com/eezstreet/OpenD2)? You might benefit from some cooperation or something.

Off topis: I hope your thing becomes as awesome as VCMI is for HoMM3. Also personally I value a compatibility mode a lot which lets you play near vanilla, but whatevs, your project, your labor. ๐Ÿ‘

Some bitmap fonts appear to cut off parts of characters

A couple of the smaller fonts have a hard time rendering some characters. This typically manifests itself with some letters seemingly missing the rightmost pixels, notably d and g appear to be missing their right pixels in font6 and ReallyTheLastSucker. Larger fonts may also be affected, but it isn't noticeable.

Validate Attribute maps behavior

D2S.StatData isn't ever modified or used properly in conjunction with item stats. This data should be read into the Player entity's stat enum map. Once it's in a map, then items can append to it.

Some things I'm unsure about:

  • do all non-static entities need a stat map (Player, Monster, etc), i.e., does the Monster stat map function the same, or do special considerations need to be made.
  • stat map should track modifications, i.e., if a stat is modified it should appear blue
  • when an item is (un)equipped will the entire map need to be recalculated?

The concern for bullet 3: For things like modded strength/dexterity or stats in the save file it's not a big issue, however this becomes an issue when two items contain the same overwriteable stat and one is removed, which would remove the stat -- I can possibly just not care about this and deal with it later or put these sort of stats in a trailing array list. This would be a problem for say 2 items with a 30% chance to do some event. Technically this should be treated as 2x30% (60%) chance.

Move mercenary gear to mercenary entitity

Hireling should be it's own entity that is a monster that has an inventory. This will make retrieving the mercenary gear less clunky. Additionally, better detection of a save file with NO mercenary is needed.

Instead of checking gameScreen.player.merc.items.items != null, it should be as simple as gameScreen.player.merc != null

Side note yes, I'm aware that the current code looks weird as gameScreen.player.merc.items.items.items -- this is due to how the data is saved in the D2S mostly for debugging purposes to keep the data grouped as in the file. This is really GameScreen.Player.MercData.MercItemData.ItemData.Array<Item> with ItemData being the same object as the D2S.ItemData so player items are read the same way as merc items.

double check item encoding stat counts

Some stats encode consecutive stats after one another to save space (e.g., max damage stats immediately follow min damage without declaring the max damage stat header [they are 2 different stats encoded as one]). Some of my info is out of date on encoding counts of stats. There are some stats that no longer encode multiple consecutive stats and opt in to use encoded parameters. I have fixed some of them, but I'll need to go back and make sure they are all correct or fix on an ongoing basis.

See ItemStatCost.txt

Test item display stats for correctness

All of the stats I've seen look fine, however it would be good to go through and test as many as possible to ensure that the stat descriptions display like they do in the original game client -- specifically where op functions are concerned.

selectable Animation with null layers

274e756 fixed a NPE that was being thrown for some null layers in monster cof animations. This check may ultimately not be needed in production, but I am documenting it to look into later as to the specifics of why/which these layers are null (the monsters looked correct in-game). Additionally, I want to look into COF.Layer.selectable so see its functionality -- maybe it's related.

SkillsPanel skill descriptions

More testing for SkillsPanel skill descriptions is needed.

  • Support compound funcs ln34+(skill('Shiver Armor'.blvl)+skill('Chilling Armor'.blvl))*par7
  • Support for all functions in SkillCalc.txt
  • Support for all functions in SkillDesc.txt
  • Check compatibility with damage calculations, etc.

Crash on select character screen if there is a character with no shield

Crash on character select screen if there is a player without a shield. This selection screen needs to be rewritten to support different player modes too depending on primary weapon. This will require finding the player's weapon class. Currently, it assumes TN 1HS and defaults shield to LIT, but LIT shield doesn't exist and should revert to null for no shield and remove the layer.

separate Player entity from save file data

Player has become a mess and some of the code needs to be rearranged. Save specific data like hotkeys, hireling and actions don't really belong in the Player entity class. Most of this data needs to be moved to GameScreen or a new container class created similar to Riiablo.client except the client in this case refers to the game client's active player (e.g., Riiablo.player). Player should essentially just be the entity itself, so equipping an item would modify the player entity state to now draw that item, and the other classes like InventoryPanel and StashPanel would use the Riiablo.player reference instead of the game entity. The other benefit of doing this is that it would probably make implementing multiplayer players easier since only a subset of their info is ever sent to the other clients anyways, thus I could reduce the number of Player constructors and special considerations for non-local player entities.

Additionally, I would really like to support doing something like viewing the items of the players in your current game (possibly of players not in your game from the lobby even). This would either mean that some of their data is downloaded to the client (so each client would contain a list of the other players' equipped items), or I separate this logic from the game code and make this it's own packet which returns the player's equipped items. I don't know how this would effect set bonuses though, so maybe those would need to be tacked onto the packet along with the player's level.

Cannot render BitmapFont text onto Texture/Pixmap instance

Cannot render BitmapFont text onto Texture/Pixmap instance. This is required for caching of the rendered text. It also should greatly increase the speed of rendering the entirely of the text in the case of longer sentences. Resolving this would involve either hacking the gdx BitmapFontCache object, implementing my own, or modifying the gdx codebase to support such operations (which mainly involves rendering the glyph textures onto one parent texture)

Improve Box2D implementation and memory usage

Entity update loop has become a bit messy since adding screen vector, which updates during Entity#act(float) but is actually occasionally required during Entity#update(float) (the names of these methods are also a bit confusing -- update(float) is supposed to update position/logic, act(float) is supposed to update animation frames).

This seems like a good opportunity to look into integrating a native physics solution like Box2D, which may improve performance and greatly simplify a lot of the problems I'm currently having with collision detection and movement. One specific optimization that may be required is wall aggregation -- walls are subdivided into 1yd squares which Box2D will prefer represented as a single body (these walls will be needed when I want to detect rooms to fade walls). This also will help add light ray casting.

One problem I've been having is that I originally assumed player movement was based with the underlying 1yd grid, but that assumption seems to be incorrect to a point. I tested this by using a character with extremely slow walk speed, walking and then casting a spell after a short enough distance where the next subtile could not have been reached. The characters stops instantly indicating that their position is not locked to the grid. I also noted that their networked position data seems to only update positions based on the grid -- so this may be a client-local behavior.

I also started looking into unit sizes, and assuming the information I was looking at was correct, they are not using axis-aligned boxes seem to be a selection of horizontal boxes (this would explain SizeX and SizeY columns in monstats2, but they are always same as far as I've seen). With Box2D, this might be better implemented as a circle with diameter of SizeX.

add support for item by-time stats

Game files support time-of-day stats that I don't think were ever implemented (or at least items with those stats don't appear to drop). I'm not going to support them for now, but maybe they can be supported later. Maybe it would be cool to add some kind of druid item that gives stats at night?

See ItemStatCost.txt

item_armor_bytime
item_armorpercent_bytime
item_hp_bytime
item_mana_bytime
item_maxdamage_bytime
item_maxdamage_percent_bytime
item_strength_bytime
item_dexterity_bytime
item_energy_bytime
item_vitality_bytime
item_tohit_bytime
item_tohitpercent_bytime
item_cold_damagemax_bytime
item_fire_damagemax_bytime
item_ltng_damagemax_bytime
item_pois_damagemax_bytime
item_resist_cold_bytime
item_resist_fire_bytime
item_resist_ltng_bytime
item_resist_pois_bytime
item_absorb_cold_bytime
item_absorb_fire_bytime
item_absorb_ltng_bytime
item_absorb_pois_bytime
item_find_gold_bytime
item_find_magic_bytime
item_regenstamina_bytime
item_stamina_bytime
item_damage_demon_bytime
item_damage_undead_bytime
item_tohit_demon_bytime
item_tohit_undead_bytime
item_crushingblow_bytime
item_openwounds_bytime
item_kick_damage_bytime
item_deadlystrike_bytime
item_find_gems_bytime

MapRenderer debug preset grid drawing grid size incorrect

drawDebugGrid in MapRenderer when RENDER_DEBUG_GRID=3 is drawing the incorrect grid size, or it's offset somehow and not aligning with the actual preset bounds. I think this is an error I made when calculating the modulus. The bug show up in the Den of Evil zone and works as expected in the default zones (zones branching from 0,0.


Double checked and yes. The bounds are calculated from 0,0 and not from the zone's origin tx,ty, so it's offset because DOE is not aligned on the same grid size as the origin.

fix modules that relied on gdx.diablo

fc5786a broke a lot of existing older modules that should be ported over where possible. Some of these were using deprecated features that the riiablo refactor removed (really, really old code that was since long redeveloped).

  • android
  • core
  • desktop
  • mapbuilder
  • mpqlib
  • server
  • mpqviewer Some things work, but a lot of it is broken, mostly because I've added features to some of the code, e.g., new Animation class throws a null pointer error because this module doesn't init some Riiablo class fields. I'll probably go through this and make sure it works properly after I add the new mpq module, but for now that is time better spent on the game itself because I can use DRTest.
  • ds1viewer This was based off of the first version of the MapRenderer which was pretty much completely junked and improved. This needs to be rewritten almost entirely. Some of it's functionality is in mapbuilder but this is still useful to look at individual DS1s. If possible I'd even like to delete this in favor of adding its functionality to mpqviewer
  • tester EXTREMELY broken. This was always a sandbox module to test codecs, features and implement one-off tools. I want to fix as much of this as I can by adding proper tests for everything in core module and maybe deleting this module and adding a tools module to contain things like my font metrics editor

desktop vs mobile UI

I'm going to do my best to update this issue with current screenshots of desktop vs mobile UI. I'll start taking ideas later on. I'm going to add some more icons on mobile once I implement more features and add a couple more skill slots -- bottom right is the interact button to talk to NPCs, enter warps, pickup items, etc -- the other 3 are skills. Very much a WIP -- also note that the camera on mobile is at 80% of desktop -- I think it looks better and helps with performance, but I may add a config option for this.

Desktop
Desktop

Mobile
Mobile

DT1 isometric tile storage optimization

I'll look into this more when loading speed improvements become more of a priority:

DT1 isometric tiles are stored as 160x80 (really 160x79) textures, however approximately 1/2 of that space is garbage due to the square textures containing a diamond/isometric texture. Two ideas I've had to far to improve this is to store the texture data as a rectangle texture and skew it somehow (idea 1) to render it correctly, or more probable (idea 2), use a static lookup texture 160x80/160x79 consisting of r,g as x,y formatted as the current textures are (rectangle containing isometric data) to a second rectangle texture containing the actual texture data. The second texture would be approx 1/2 the original size at about 80x80 and greatly ease the desire to pack all textures from a given DT1. Only 1 copy of the static texture is needed, so overhead is negligible.

I think idea 2 is doable and the performance overhead minimal for the space/loading time efficiency it will provide (1/2 the data). Additionally, some DT1s contain an upwards of 300+ tiles, thus an upwards of a 20x20 sheet goes from 3200x1600 to 1600x1600. This might require a second shader, so some of the features of the first will need to be ported or generalized an applied to some FBO post shader (namely gamma).

I don't know if this will be worth the trouble either, I'll need to benchmark exactly how much DT1 data is loaded in-game (looks to be 1.5MB per DT1 on the high end), and many DT1s contain both isometric and RLE tiles, so packing could become an issue, however it's plausible to optimize the RLE tiles as well and store them as a 32x32 grid atlas (?1 atlas for iso tiles, another for rle tiles?) with those tiles containing coords and offsets to the atlas positions they need (this could provide a nice storage efficiency improvement as well, since there's a lot of wasted space in some RLE tiles).

Total act 1 DT1s are only roughly 27.205 MB, so assume 32 MB of DT1 data per act. I'll also benchmark more when I look into streaming the data and optimize to avoid asset loading lockups. I think the MPQs/Asset Manager are the major data-access holdups right now (the data itself is all pretty tiny) - and Android is the only place where I think this DT1 change would have any notable effect.

FlatBuffers gradle setup

f361d48 added support for FlatBuffers, however the gradle config needs to be fixed, and there doesn't seem to be much support.

  • Current build process assumes flatc is on the system's path -- should this be included within the repo? My instinct is yes to ensure flatc.exe is the same version as the gradle version
  • createFlatBuffers task compiles schemas to java sources in gen/ -- should this be within build/generated and not included within repo? (this would mean that people trying to build the sources would need to compile the schemas to generate the java sources and then have those compiled to classes)
  • gen/ is not automatically detected as Generated Sources Root by IntelliJ IDEA -- I configured this as one of the source sets because I want the compiler to compile these as well, however idea.module.generatedSourceDirs doesn't seem to be working unless I am misunderstanding

Anyways, I need to figure this stuff out. In the meantime, as I hash out the FlatBuffer schemas, I think I will include their generated java sources in the repo as needed until I can figure out the correct procedure.

add support for warp highlight tiles

Some (if not all) warps have special tiles that are highlighted versions of existing tiles (e.g., normal cave entrance is highlighted when cursor is over it). The sub index of the special warp tile tells us the main index of the affected tiles (both highlighted and non-highlighted tiles have the same main index). In all cases I've seen, either 2 or 4 tiles are replaced with either sub index 0-1 -> 2-3 or 0-3 -> 4-7. The trick is determining which is the case. Also, is it always the case where replacement tiles are available?

Possibly useful columns in LvlWarp.txt are LitVersion, Tiles, Direction. Based on previous knowledge, Tiles was supposed to determine some kind of offset, this is false from what I've seen, since I've seen 2 and 4 tile variants both with Tiles=2, and the only time Tiles!=2 is for 3 expansion barricades where it is 4.

Also, sometimes there are 2 special warp tiles for the same warp. I think when this is the case, there is a 1:1 mapping of the special warp tiles with the replacement tiles (so this marks the 2 tile case), so if there is only 1, I think this marks the 4 tile case.

I'm going to investigate a bit more, because with the current implementation, the 2 special warp tile case will create 2 warp entities. I'm also not sure if the original client uses entities for warps like I am (my method will work fine though).

I also need to add some logic to MapRenderer to draw the (non)highlighted tiles correctly. Currently my PopPads implementation bypasses using entities and is implemented into MapRenderer itself. I think rendering the highlight tiles is more appropriate from within MapRenderer than the entity itself, not to mention the complications it will provide with the order of the rendering if done within the warp entity. The question is how I want to handle tying the Warp entity with the tiles it hides/unhides in MapRenderer

Make entity angles consistent

Entity angles are currently configured as relative to pixel location. This is becoming an issue and I think it's about time to change this behavior to map the angle to the in-game map grid instead (0=east, pi=west, pi/2=north, -pi/2=south). I think this will simplify some of the common operations (e.g., look at an entity).

See Entity#lookAt(Entity) or Entity#lookAt(float,float) for just how convoluted the current implementation is -- i.e., every time you want to look at a position it needs to be converted to the pixel location and then mapped to the angle from there.

The primary reason why this change should be looked into is that Missile entities require a vector target and the method to calculate this correctly is crazier than the methods I mentioned above to do something that should be fairly simple. I will need to look at some of the other Missile travel functions as well to see some of the other patterns (e.g., blessed hammer travels in a spiral, is this relative to in-game map grid?).

In other words, I think the current implementation is really the corner case and the default implementation should be relative to map grid.

improve set item detection

Set item detection is a bit hacky. It would be beneficial to create some kind of enumeration of the sets, possibly using Sets.txt. I think each set needs a bitsum of held items which can get set when the player is loaded and updated when items are picked up/dropped. It would also be nice to iterate through the Sets.txt and generate the set item lists.

Transmog support

I'd like to add transmog support for at least some item types (weapons/armor). Customization might include changing the invfile, flippyfile, color, trans. Some items will need to remain within their families, e.g., no changing a 2HS to a 1HS. I'm unsure of how this info can be saved in addition to the D2S, either support for D2S is dropped while retaining the ability to import, or this data will need to be saved in addition to (probable). I'll look into this more somewhere down the line, likely when the game is largely finished.

Display socketed items when item is focused

I added a note for this in Item. Sockets should display in the correct order and this needs to be consistent with the original client. This is based on number of cells high an item is, e.g., armor shows 3 sockets in a vertical line, shields with only 2 cells high shows sockets in a triangle pattern. The socketed item images need to be loaded with the item image.

aggregating stats versus additional stats

Aggregations seem to be performed on a list-by-list basis, i.e., runeword properties will add to existing magic properties, but set properties will not aggregate with magic and (I assume, but untested) rune properties.

It seems that multiple copies of the same aggregateable skill in a single list will choose the last one. Some skills maintain their independence, e.g., charged skills will not overwrite if it's the same skill with a different level (i.e., different param value), but if the level and skill are the same (i.e., same param value) the skill is overwritten. In this case, the charges will add to each other.

I need to enumerate this relationship and investigate more, but my best guess is:

  • property lists should be implemented as a map of stat instances
  • if the property has param, this should be included in to the stat hash somehow, since the value in this case determines the number of charges, level of the skill, etc, but param determines which skill, etc.
  • adding should be performed on a per-encoding basis -- side note: I haven't tested this, but can the added stats overflow in the original game, or do the encoded values get extended into 32-bit ints?
  • inherent item property list stats instances should probably be mutable
  • when generating the details graphic, set properties shouldn't aggregate
  • when saving property lists when the item is written to the d2s lists should be saved separately

To reiterate on the property lists:

  • item inherent properties base item stats as an aggregate of all below active list properties pre-calculated (i.e., def with ED applied, damage with damage mods applied, etc) I'm not sure if this is the correct strategy, but it makes sense.
  • magical properties
  • set properties 1-5 only one of these can be active at a time, does not aggregate with other lists
  • runeword properties
  • display properties similar to inherent properties, except used only for displaying the properties, i.e., this property list might contain fake properties that aggregate other property lists and join them (so where in other lists resistances are displayed discretely, this might join them as + to all resistances). I think this will get generated with the item details and updating the item details (charge used, durability changed, etc) will re-generate this property list.

So, item inherent properties list will be all the active list values added together, and then the algorithm will run to modify the item values all at once, so all the properties ED will add together and then when the lists are all added, ED will actually be calculated and applied to defense stat, etc.

TODO: how do things like aggregated charges work, since I'm guessing it subtracts a charge from one of the properties internally and then maybe re-generates the aggregate stat.
TODO: I need to figure out and more clearly define the differentiation between the visual list aggregation and the actual item property aggregation, since all the item stats need to continue to be discrete internally for saving purposes, but I don't think it's realistic to calculate them on the fly every time a stat is retrieved at least visually

DT1 resource asset streaming

All DT1s for a given act are currently loaded before the Map instance is even built (Map dependencies need to be declared before MapLoader is called, and since MapLoader builds the Map instance, we don't know what those are until after it's called). I think this approach is a bit too intensive for mobile, since at least on my device, loading takes ~12-15 seconds (I'm only using a subset of an act's DT1s to avoid this issue right now to get me ~6-7 seconds load time). I would like to look into the possibility of streaming these assets more efficiently or add hidden loading screens (fade in from black) to hide the loading. I think it's plausible to make the loading screen the bare minimum assets, e.g., the subset of DT1s that belong in the current level type (i.e., if you switch to a different act waypoint, load that level type DT1s first over the town's), but when building the Map instance, the required DT1s must be loaded into memory since they contain stuff like walkable tile flags.

I'd also like to note that the current loading method would likely benefit largely by #8 and even #7 since each area's main DT1 is about 2 MB.

Improve spell auto add/remove

eca930a removed the hacky top/bottom rows in the spell quick panel in favor of defining the default skill set (attack, kick, etc). Two things that need to be mediated are

  1. certain default skills are conditionally added (if weapon is throwable, then add throw/left hand throw. I need to check and see the behavior of left hand throw versus throw and also if attack is conditional in some case (e.g., maybe it's not available if the character can't attack for some reason).
  2. charged skills such as those given by tome of (identify|town portal) and also scroll variants need to be added dynamically. This means adding support for detecting inventory items in addition to charms. I will need to check for built-in support in the excels for adding these skills.

It's possible that the item types somehow define the skills they add, so I'll look into this as well.

How to determine warp pairs

Right now the pairs are simply hard-coded during generation. I can expand upon this by defining which IDs refer to entrances/exits, but this needs to cover cases where a level can have multiple entrances/exits (see palace cellar/canyon of the magi.

MPQ read speedups -- possible native code required

I've noticed there are some slowdowns when reading files from MPQs, specifically pkexplode. I tried changing readBytes to read as many sectors at once (up to the uncompressed sector size) thinking this was one of the major issues (a compressed sector is only about 1500ish bytes, so I can read 3-4 at a time and be under 4K), but that didn't have as large of an effect as I had hoped. I also have looked into other file reading strategies, but I don't think they will provide a significant increase.

This is an issue on android where monstats.txt alone (90K compressed, 432K uncompressed) is taking 3-4 seconds to read, completely stalling the application start -- as an intermediate fix, I will likely add multi-threaded support to read the initial txt files in the background (MPQ reads are single-threaded). This will require a new AssetManager because it's only single-threaded, but I was planning on that anyways because of this exact reason -- I'd like a way to pre-load things in the background of AssetManager, since stuff like the TXTs are used extensively in-game and can be pre-loaded while the player is starting the app, selecting a character and joining a game without locking up assets that are needed ASAP. I can also implement support for bin files for at least a few of the files, since bin files are smaller and significantly faster to read.

I am considering transitioning the MPQ lib to its own module and implementing at least selective native code support for some of the more intensive operations (decompression and exploding), since those appear to be the biggest holdups, and I think handling them with native code might be something to look into.

Add support for D2S creation (character creation)

d485878 broke character "creation" because I removed support for creating fake characters. Before, created characters would just create a player entity with the correct character and stub out the stats. All characters now require a D2S whether that be a form of a save file or a stub D2S object. Frankly it's about time character creation was added in properly to at least generate a D2S object.

I don't want to add support for writing an actual D2S file until later when most of the game features work (quests, item drops, etc). People who test this need to use their own existing characters -- I may even upload mine.

TextWidget does not render glyphs

TextWidget is not rendering glyphs for the text it represents. The rest of the Widget is being rendered correctly (e.g., debug bounds).

Excel class API changes

I want to look into changing some functionality of Excel and improving it. Some of these are tentative

  • remove reliance on TXT/optimize -- TXT isn't used directly anywhere, so I want to look into if any optimizations can be made in the excel loading if I don't have to go through TXT first, however it might be the case that TXT can be interchangeable with BIN when that is added
  • add support for BIN files which can be parsed faster than TXT and contain a subset of columns (namely removing the metadata columns not needed for a release version)
  • foreign key support -- flag columns as foreign keys which can be assigned at a later time (second pass of the parser once the other tables have loaded). This is an extremely common use-case, so it might improve readability if this were the case that I could simply replace these columns with references to the corresponding data, instead of the index of that data. I want to avoid tightly coupling the tables with Riiablo.files if I can though.
  • transition from cascading if-statements to adding a map indexed on type with parsers for the excel columns. I think this will improve speed and extensibility, but I also want to avoid auto-boxing the data wherever possible
  • it might be nice to support parsing data structures, e.g., some excel files contain multiple numbered columns (stat1, param1, stat2, param2), but I think it might be nice if this could be parseable into stat[] object consisting of a stat, param instead of stat[], param[], this just keeps associated data together
  • related to above point, it would be nice to support multi-dimensional arrays, or at least 2D. E.g., some stats have copies for each difficulty, and partial set bonuses would be easier to implement in a 2D array.

This might be doable when some #8 improvements on loading TXT files are made.

propagate stat modifications to maxstat (if exists)

acdf053 provided a temp fix to the discrepancy between 3/4 stats that have a maxstat (maxdurability remains unfixed). I believe the intention of this column is to indicate an additional stat which operations on the base stat should also be preformed on (as well as serving as the max value). The current fix is insufficient and hacky for cases where the player loads with less than max health. A bit more investigation is needed for all of the use-cases for this column.

See direct column as well -- looks like it indicates existence of maxstat.

Direct - this boolean controls whenever this stat has a maximum stat associated with it, which among others means that unless you screw up something badly, this stat can never exceed the value of the maximum stat and that any change to this stat is permanent (a skill that alters mana, hitpoints or stamina is permanent, while a skill that alters maxmana, maxhp and maxstamina is temporary). You do not need to specify a maximum stat for the effect to be permanent, this can be used to get kill counters to work (look for SVR's old post).

MaxStat - the maximum stat associated with this stat, this field only ever has an effect if Direct is set to true, see the discussion under that field. Percentage based increases to the max stat will usually alter this stat too, however this has some hardcoded aspects to it and is not entirely softcoded.

Draw FPS GlyphLayout empty in GameScreen

1590adc fixed an issue where setting text on a GlyphLayout obtained through Pools.obtain(GlyphLayout.class) would not create any runs. I worked around the behavior by using a GlyphLayout reference that is held onto for the life of the application, however I would prefer to use the original method and I don't understand why it was not working as expected. I'm not sure if this is a bug with my application or with LibGDX. Overall this issue is low impact, but I'm concerned it is further-reaching that what I've noticed.

This only happened when the GlyphLayout was obtained while the application was in GameScreen and worked within the menus, etc, so maybe a reference isn't being released properly?

animation with subset of frames

This came up while creating the quest buttons in QuestsPanel. Some DC6 use a subset of the frames for a direction for an animation. In the case of the quest buttons, 0-24 are the quest finished animation, while 25,26 are state animations for press and disable.

audio stops working

Sometimes the audio stops working after an extended period. I can't reproduce this, but it happened after I left the app running overnight. Documenting for later investigation.

StringTBL duplicate keys + performance improvement

I noticed that strModEnhancedDamage contains two references in patch_d2.mpq -- data\local\lng\eng\patchstring.tbl which is causing an issue because the result from the first key is returned, which is incorrect. The first key ends with a new line character while the second key does not. From what I've seen in item mods, the first key should be junk and the second key is what we actually want.

In the meantime I've disabled that specific problem key within StringTBLs by changing its strOffset--, which effectively reduces the key's string length by 1 and means that when comparing the key length is will only contain strModEnhancedDamag and no longer match, so the second key will return as expected. In other words, I've broken the first key on purpose.

Long term I would like to (and I think this is representative in the original design), search for existing keys when a StringTBL is loaded and basically overwrite them/disable existing keys which brings me to my second point:

Looking at how StringTBL lookups are performed, it might be easier to compare the keys using their hashes instead of comparing their content (or make the hash comparison a fail-first check). Some tests may need to be performed to see if all the keys are unique, or unique enough, but this may provide a decent performance boost if they are indeed unique. I have seen cases though where the lookup needs to iterate through the table a bunch, so this may mean all those iterations are of keys with equivalent hashes, meaning my approach is misguided.

License?

Which license type this project use?
I want to add this to the https://osgameclones.com/ but we need to add the license and I don't see it in this repository.

dual wielding animations missing layers/components

Selecting LH components list for RH only animation results in a null layer. Temp fix referenced below checks for null layer and ignores.

Some layers are missing -- e.g., left hand swing does not contain any weapons in the right hand (appears to be HTH only is added for RH).

I think both of these issues are related and additional logic might be required for adding the missing layers for dual wielding cases (see assassin dual wield claws also). See 3b9311b

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.