The batchcrypt from marcoszh

Cross-Silo Vertical Learning?

I understand that for cross-silo horizontal learning settings, what need to be encrypted are gradients from different data owners, then performing BatchCrypt means HE + "local batch norm", and it will hold true for an acceptable loss of precision, and is not likely to harm convergence and performance of the model.

Do you have any idea on how this could be done in a cross-silo vertical learning setting? since the intermediate results are not only gradients, but linear computation result, as well as a component used for computing the gradient. Applying "BatchNorm" seems doesn't make sense on these, any suggestions?

"2021-10-27 08:52:17,190 - task_scheduler.py[line:228] - INFO: job 20211027085214806711127 component dataio_0 run on guest 10000 status is notRunning"
"2021-10-27 08:52:17,191 - task_scheduler.py[line:228] - INFO: job 20211027085214806711127 component dataio_0 run on host 10000 status is notRunning"
"2021-10-27 08:52:17,954 - job_controller.py[line:123] - INFO: job 20211027085214806711127 component dataio_0 host 10000 status running"
"2021-10-27 08:52:17,979 - job_controller.py[line:123] - INFO: job 20211027085214806711127 component dataio_0 guest 10000 status running"
"2021-10-27 08:52:18,063 - job_controller.py[line:123] - INFO: job 20211027085214806711127 component dataio_0 host 10000 status failed"
"2021-10-27 08:52:18,113 - job_controller.py[line:123] - INFO: job 20211027085214806711127 component dataio_0 guest 10000 status failed"
"2021-10-27 08:52:18,193 - task_scheduler.py[line:228] - INFO: job 20211027085214806711127 component dataio_0 run on guest 10000 status is failed"
"2021-10-27 08:52:18,194 - task_scheduler.py[line:228] - INFO: job 20211027085214806711127 component dataio_0 run on host 10000 status is failed"
"2021-10-27 08:52:18,194 - task_scheduler.py[line:154] - INFO: job 20211027085214806711127 component dataio_0 run failed"
"2021-10-27 08:52:19,885 - job_controller.py[line:196] - INFO: job 20211027085214806711127 on guest 10000 start to clean"
"2021-10-27 08:52:19,889 - job_controller.py[line:202] - INFO: job 20211027085214806711127 component dataio_0 on guest 10000 clean done"
"2021-10-27 08:52:19,889 - job_controller.py[line:207] - INFO: job 20211027085214806711127 on guest 10000 clean done"
"2021-10-27 08:52:20,911 - job_controller.py[line:196] - INFO: job 20211027085214806711127 on host 10000 start to clean"
"2021-10-27 08:52:20,915 - job_controller.py[line:202] - INFO: job 20211027085214806711127 component dataio_0 on host 10000 clean done"
"2021-10-27 08:52:20,916 - job_controller.py[line:207] - INFO: job 20211027085214806711127 on host 10000 clean done"
"2021-10-27 08:52:21,985 - job_controller.py[line:196] - INFO: job 20211027085214806711127 on arbiter 10000 start to clean"
"2021-10-27 08:52:21,986 - job_controller.py[line:207] - INFO: job 20211027085214806711127 on arbiter 10000 clean done"
"2021-10-27 08:52:21,988 - task_scheduler.py[line:111] - INFO: job 20211027085214806711127 finished, status is failed"

In ERROR.log (/dataio_0 for guest)


- [ ] "2021-10-27 08:52:17,992 - task_executor.py[line:120] - ERROR: '_DTable' object has no attribute 'get_meta'"
- [ ] Traceback (most recent call last):
- [ ]   File "/data/projects/fate/python/fate_flow/driver/task_executor.py", line 109, in run_task
- [ ]     run_object.run(parameters, task_run_args)
- [ ]   File "/data/projects/fate/python/federatedml/model_base.py", line 157, in run
- [ ]     self._run_data(args["data"], stage)
- [ ]   File "/data/projects/fate/python/federatedml/model_base.py", line 131, in _run_data
- [ ]     self.data_output = self.fit(data)
- [ ]   File "/data/projects/fate/python/federatedml/util/data_io.py", line 736, in fit
- [ ]     return self.reader.read_data(data_inst, "fit")
- [ ]   File "/data/projects/fate/python/federatedml/util/data_io.py", line 119, in read_data
- [ ]     self.generate_header(input_data, mode=mode)
- [ ]   File "/data/projects/fate/python/federatedml/util/data_io.py", line 83, in generate_header
- [ ]     header = input_data.get_meta("header")
- [ ] AttributeError: '_DTable' object has no attribute 'get_meta'
- [ ]

IN ERROR.log (/dataio_0 for host)

"2021-10-27 09:32:17,221 - task_executor.py[line:120] - ERROR: Count of data_instance is 0"
Traceback (most recent call last):
  File "/data/projects/fate/python/fate_flow/driver/task_executor.py", line 109, in run_task
    run_object.run(parameters, task_run_args)
  File "/data/projects/fate/python/federatedml/model_base.py", line 157, in run
    self._run_data(args["data"], stage)
  File "/data/projects/fate/python/federatedml/model_base.py", line 131, in _run_data
    self.data_output = self.fit(data)
  File "/data/projects/fate/python/federatedml/util/data_io.py", line 736, in fit
    return self.reader.read_data(data_inst, "fit")
  File "/data/projects/fate/python/federatedml/util/data_io.py", line 115, in read_data
    abnormal_detection.empty_table_detection(input_data)
  File "/data/projects/fate/python/federatedml/util/abnormal_detection.py", line 25, in empty_table_detection
    raise ValueError("Count of data_instance is 0")
ValueError: Count of data_instance is 0

why this error is occurred and How can I solve this problem?

marcoszh / batchcrypt Goto Github PK

batchcrypt's People

Contributors

Stargazers

Watchers

Forkers

batchcrypt's Issues

Cross-Silo Vertical Learning?

configuration environment?

import error

[ERROR] 1.0.1-final _DTable object has no attribute "get_meta"

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent