Commit c87e42bb authored by Litao Jiao's avatar Litao Jiao
Browse files

net/smc: use read semaphores to reduce unnecessary blocking in smc_buf_create() & smcr_buf_unuse() 

mainline inclusion
from mainline-v6.3-rc1
commit f6421014
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I7809T
CVE: NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/net/smc?id=f6421014e88983c5bb7a25c71c01ae6278a01df9



--------------------------------

Following is part of Off-CPU graph during frequent SMC-R short-lived
processing:

process_one_work				(51.19%)
smc_close_passive_work			(28.36%)
	smcr_buf_unuse				(28.34%)
	rwsem_down_write_slowpath		(28.22%)

smc_listen_work				(22.83%)
	smc_clc_wait_msg			(1.84%)
	smc_buf_create				(20.45%)
		smcr_buf_map_usable_links
		rwsem_down_write_slowpath	(20.43%)
	smcr_lgr_reg_rmbs			(0.53%)
		rwsem_down_write_slowpath	(0.43%)
		smc_llc_do_confirm_rkey		(0.08%)

We can clearly see that during the connection establishment time,
waiting time of connections is not on IO, but on llc_conf_mutex.

What is more important, the core critical area (smcr_buf_unuse() &
smc_buf_create()) only perfroms read semantics on links, we can
easily replace it with read semaphore.

Signed-off-by: default avatarD. Wythe <alibuda@linux.alibaba.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
Signed-off-by: default avatarLitao Jiao <jiaolitao@sangfor.com.cn>
parent 9dded3bb

net/smc/smc_core.c

+4 −4
Original line number Diff line number Diff line
@@ -614,10 +614,10 @@ static void smcr_buf_unuse(struct smc_buf_desc *rmb_desc, 
		rc = smc_llc_flow_initiate(lgr, SMC_LLC_FLOW_RKEY);

		if (!rc) {

			/* protect against smc_llc_cli_rkey_exchange() */

			down_write(&lgr->llc_conf_mutex);

			down_read(&lgr->llc_conf_mutex);

			smc_llc_do_delete_rkey(lgr, rmb_desc);

			rmb_desc->is_conf_rkey = false;

			up_write(&lgr->llc_conf_mutex);

			up_read(&lgr->llc_conf_mutex);

			smc_llc_flow_stop(lgr, &lgr->llc_flow_lcl);

		}

	}
@@ -1587,7 +1587,7 @@ static int smcr_buf_map_usable_links(struct smc_link_group *lgr, 
	int i, rc = 0, cnt = 0;



	/* protect against parallel link reconfiguration */

	down_write(&lgr->llc_conf_mutex);

	down_read(&lgr->llc_conf_mutex);

	for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {

		struct smc_link *lnk = &lgr->lnk[i];
@@ -1600,7 +1600,7 @@ static int smcr_buf_map_usable_links(struct smc_link_group *lgr, 
		cnt++;

	}

out:

	up_write(&lgr->llc_conf_mutex);

	up_read(&lgr->llc_conf_mutex);

	if (!rc && !cnt)

		rc = -EINVAL;

	return rc;