[Bug] TruChain with AgentExecutor breaks LangChain about trulens HOT 6 CLOSED

Thanks for report. Take a look at #376 for fix.

from trulens.

Tested this with trulens-eval 0.10.0, it doesn't break anymore but it prints a couple warnings:

A new object of type <class 'langchain.chains.llm_math.base.LLMMathChain'> at 0x7ff89668ef00 is calling an instrumented method <function Chain.__call__ at 0x7ff8a74ab5e0>. The path of this call may be incorrect.
Guessing path of new object is *.app based on other object (0x7ff8965c0300) using this function.
A new object of type <class 'langchain.chains.llm.LLMChain'> at 0x7ff8974d86c0 is calling an instrumented method <function Chain.__call__ at 0x7ff8a74ab5e0>. The path of this call may be incorrect.
Guessing path of new object is *.app based on other object (0x7ff8965c0300) using this function.

Should that be Ok?

from trulens.

Hi, this is happening because our walk over the agent is not finding all of the relevant components that need instrumentation but at the same time it has walked over and instrumented other components that have the same class/methods that the undiscovered components use. Because of this, we cannot be sure where in the app structure these components are located when we see them invoking methods. If you do not care what their location is listed as in the dashboard or otherwise, this may be ok for you. If you do care about the paths, this is not ok. I am investigating the enumeration of agent components to address this.

from trulens.

Investigating further I see this may all be because the components are hidden behind a method (the func attribute of Tool) which we do not inspect when instrumenting components. Will think about how to address this but there is no current plan to fix the path of components that appear only inside closures like this.

from trulens.

I am adding more agent components to instrument that helps a little with this issue but is not perfect (#406). Things like Tool._run now show up in the dashboard with their correct path (see first screenshot), but the LLM call involved in the tool run that is performed by Tool.func has the wrong path (see second screenshot).

from trulens.

logging message issue is fixed. Expanded topic is ok as is

from trulens.