A Privacy-Preserving Scheme for Multi-Party Vertical Federated Learning

doi:10.12142/ZTECOM.202404012

ZTE Communications ›› 2024, Vol. 22 ›› Issue (4): 89-96.DOI: 10.12142/ZTECOM.202404012

• Research Papers • Previous Articles

A Privacy-Preserving Scheme for Multi-Party Vertical Federated Learning

FAN Mochan¹, ZHANG Zhipeng¹(), LI Difei¹, ZHANG Qiming²^,³, YAO Haidong²^,³

^1.School of Information & Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
^2.ZTE Corporation, Shenzhen 518057, China
^3.State Key Laboratory of Mobile Network and Mobile Multimedia Technology, Shenzhen 518055, China

Received:2023-11-14 Online:2024-12-20 Published:2024-12-03
About author:FAN Mochan is a PhD candidate at University of Electronic Science and Technology of China (UESTC). She received her BS degree from Suzhou University of Science and Technology, China and MS degree from Jiangxi University of Science and Technology, China. Her research interests include network security, blockchain, and federated learning.
ZHANG Zhipeng (2474297092@qq.com) is pursuing a master's degree at University of Electronic Science and Technology of China (UESTC). He received his bachelor's degree from UESTC. His research interests include network security and distributed machine learning.
LI Difei is pursuing a master's degree in communication and information system at University of Electronic Science and Technology of China. His research focuses on machine learning.
ZHANG Qiming is a senior system architect at ZTE Corporation. He received his bachelor's degree from Zhejiang University, China in 1992. His research interests include MEC and heterogeneous computing.
YAO Haidong is a senior system architect at ZTE Corporation. He is engaged in the research and design of deep learning, large model network architecture, and compilation conversion technology.
Supported by:
ZTE Industry?University?Institute Cooperation Funds(202211FKY00112);Open Research Projects of Zhejiang Lab(2022QA0AB02);Natural Science Foundation of Sichuan Province(2022NSFSC0913)

Abstract

Abstract:

As an important branch of federated learning, vertical federated learning (VFL) enables multiple institutions to train on the same user samples, bringing considerable industry benefits. However, VFL needs to exchange user features among multiple institutions, which raises concerns about privacy leakage. Moreover, existing multi-party VFL privacy-preserving schemes suffer from issues such as poor reliability and high communication overhead. To address these issues, we propose a privacy protection scheme for four institutional VFLs, named FVFL. A hierarchical framework is first introduced to support federated training among four institutions. We also design a verifiable replicated secret sharing (RSS) protocol $32$ -sharing and combine it with homomorphic encryption to ensure the reliability of FVFL while ensuring the privacy of features and intermediate results of the four institutions. Our theoretical analysis proves the reliability and security of the proposed FVFL. Extended experiments verify that the proposed scheme achieves excellent performance with a low communication overhead.

Key words: vertical federated learning, privacy protection, replicated secret sharing

FAN Mochan, ZHANG Zhipeng, LI Difei, ZHANG Qiming, YAO Haidong. A Privacy-Preserving Scheme for Multi-Party Vertical Federated Learning[J]. ZTE Communications, 2024, 22(4): 89-96.

Figures/Tables 8

Table 1 Description of the symbols involved in this paper

Symbol	Description
$P 0$	Active participant (holding labels)
$P i$ , $i ∈ 1, 2, 3$	Passive participants
$x$ , $y$ , $z$	Secrets held by $P 1$ , $P 2$ and $P 3$ respectively
$x i$ , $y i$ , $z i$	Secret shares
$H (·)$	One-way hash function

Table 1 Description of the symbols involved in this paper

Symbol	Description
$P 0$	Active participant (holding labels)
$P i$ , $i ∈ 1, 2, 3$	Passive participants
$x$ , $y$ , $z$	Secrets held by $P 1$ , $P 2$ and $P 3$ respectively
$x i$ , $y i$ , $z i$	Secret shares
$H (·)$	One-way hash function

Table 2 Parameters and their default values

Parameter	Description	Value	Default
F	Number of features	{10, 20, 30, 40}	10
T	Number of trees	{3, 4, 5, 6, 7}	3
D	Number of depths	{3, 4, 5, 6, 7}	3
I	Number of data samples	{5k, 10k, 15k, 20k, 25k}	25k

Table 3 Model performance comparison under different parameters

		GiveMeSomeCredit						UCI Credit Card
		FVFL			MP-FedXGB			FVFL			MP-FedXGB
		ACC	F1	AUC	ACC	F1	AUC	ACC	F1	AUC	ACC	F1	AUC
T	3	0.933 3	0.245 7	0.827 0	0.930 6	0.213 1	0.719 4	0.823 2	0.466 7	0.752 5	0.822 5	0.458 8	0.768 6
	4	0.933 9	0.244 2	0.829 5	0.932 6	0.235 8	0.746 5	0.824 2	0.447 2	0.760 9	0.823 7	0.475 2	0.770 1
	5	0.934 2	0.247 4	0.829 9	0.933 4	0.292 9	0.749 7	0.825 8	0.462 0	0.763 4	0.824 9	0.473 8	0.773 5
D	3	0.933 3	0.245 7	0.827 0	0.930 6	0.213 1	0.719 4	0.823 2	0.466 7	0.752 5	0.822 5	0.458 8	0.768 6
	4	0.934 7	0.280 5	0.828 6	0.931 8	0.201 4	0.737 9	0.824 4	0.451 8	0.762 8	0.823 9	0.446 1	0.769 0
	5	0.935 4	0.278 2	0.847 9	0.932 2	0.264 6	0.738 3	0.824 8	0.453 4	0.764 7	0.824 3	0.454 6	0.770 7

References 18

1	KAIROUZ P, MCMAHAN H B, AVENT B, et al. Advances and open problems in federated learning [J]. Foundations and trends^® in machine learning, 2021, 14(1–2): 1–210. DOI: 10.1561/2200000083
2	YANG Q, LIU Y, CHEN T J, et al. Federated machine learning: concept and applications [J]. ACM transactions on intelligent systems and technology (TIST), 2019, 10(2): 1–19. DOI: 10.1145/3298981
3	CHOUDHURY A, PATRA A. Secure multi-party computation against passive adversaries [M]. Cham: Springer International Publishing, 2022. DOI: 10.1007/978-3-031-12164-7
4	OU W, ZENG J H, GUO Z J, et al. A homomorphic-encryption-based vertical federated learning scheme for rick management [J]. Computer science and information systems, 2020, 17(3): 819–834. DOI: 10.2298/csis190923022o
5	WANG C, LIANG J, HUANG M, et al. Hybrid differentially private federated learning on vertically partitioned data [J]. 2020. DOI: 10.48550/arXiv.2009.02763
6	HE D J, DU R M, ZHU S S, et al. Secure logistic regression for vertical federated learning [J]. IEEE Internet computing, 2022, 26(2): 61–68. DOI: 10.1109/MIC.2021.3138853
7	NI X, XU X L, LYU L J, et al. A vertical federated learning framework for graph convolutional network [EB/OL]. (2021-06-22)[2023-06-13].
8	YANG S W, REN B, ZHOU X H, et al. Parallel distributed logistic regression for vertical federated learning without third-party coordinator [EB/OL]. (2019-11-22)[2023-06-13].
9	LI Q B, WU Z M, CAI Y Z, et al. FedTree: a federated learning system for trees [C]//Proc. Machine Learning and Systems 5. MLSys, 2023
10	XIE L C, LIU J Q, LU S T, et al. An efficient learning framework for federated XGBoost using secret sharing and distributed optimization [J]. ACM transactions on intelligent systems and technology, 2022, 13(5): 1–28. DOI: 10.1145/3523061
11	XU R H, BARACALDO N, ZHOU Y, et al. FedV: privacy-preserving federated learning over vertically partitioned data [C]//Proc. 14th ACM Workshop on Artificial Intelligence and Security. ACM, 2021. DOI: 10.1145/3474369.3486872
12	YANG K, FAN T, CHEN T, et al. A quasi-newton method based vertical federated learning framework for logistic regression [EB/OL]. (2019-12-01)[2023-07-11].
13	FANG W J, ZHAO D R, TAN J, et al. Large-scale secure XGB for vertical federated learning [C]//Proc. 30th ACM International Conference on Information & Knowledge Management. ACM, 2021. DOI: 10.1145/3459637.3482361
14	CHEN W J, MA G Q, FAN T, et al. Secureboost+: a high performance gradient boosting tree framework for large scale vertical federated learning[EB/OL]. (2021-10-21)[2023-07-12].
15	SHI H R, XU Y H, JIANG Y L, et al. Efficient asynchronous multi-participant vertical federated learning [J]. IEEE transactions on big data, 2024, 10(6): 940–952. DOI: 10.1109/TBDATA.2022.3201729
16	WU Y C, CAI S F, XIAO X K, et al. Privacy-preserving vertical federated learning for tree-based models [J]. Proceedings of the VLDB endowment, 2020, 13(12): 2090–2103. DOI: 10.14778/3407790.34078112020 .
17	HUANG Y M, FENG X Y, WANG W W, et al. EFMVFL: an efficient and flexible multi-party vertical federated learning without a third party [EB/OL]. (2022-01-17)[2023-07-15].
18	ABSPOEL M, DALSKOV A, ESCUDERO D, et al. An efficient passive-to-active compiler for honest-majority MPC over rings [C]//International Conference on Applied Cryptography and Network Security. ACNS, 2021: 122–152. DOI: /10.1007/978-3-030-78375-4_6

[1]	YAN Yuna, LIU Ying, NI Tao, LIN Wensheng, LI Lixin. Content Popularity Prediction via Federated Learning in Cache-Enabled Wireless Networks [J]. ZTE Communications, 2023, 21(2): 18-24.
[2]	LIU Qinbo, JIN Zhihao, WANG Jiabo, LIU Yang, LUO Wenjian. MSRA-Fed: A Communication-Efficient Federated Learning Method Based on Model Split and Representation Aggregate [J]. ZTE Communications, 2022, 20(3): 35-42.

A Privacy-Preserving Scheme for Multi-Party Vertical Federated Learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 18

Related Articles 2

Recommended Articles

Metrics