Commit babf0bb9 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'xfs-5.19-for-linus' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux 

Pull xfs updates from Dave Chinner:
 "This is a big update with lots of new code. The summary below them
  all, so I'll just touch on teh higlights. The two main new features
  are Large Extent Counts and Logged Attribute Replay - these are two
  new foundational features that we are building more complex future
  features on top of.

  For upcoming functionality, we need to be able to store hundreds of
  millions of xattrs per inode. The Large Extent Count feature removes
  the limits that prevent this scale of xattr storage, and while we were
  modifying the on disk extent count format we also increased the number
  of data extents we support per inode from 2^32 to 2^47.

  We also need to be able to modify xattrs as part of larger atomic
  transactions rather than as standalone transactions. The Logged
  Attribute Replay feature introduces the infrastructure that allows us
  to use intents to record the attribute modifications in the journal
  before we start them, hence allowing other atomic transactions to log
  attribute modification intents and then defer the actual modification
  to later. If we then crash, log recovery then guarantees that the
  attribute is replayed in the context of the atomic transaction that
  logged the intent.

  A significant chunk of the commits in this merge are for the base
  attribute replay functionality along with fixes, improvements and
  cleanups related to this new functioanlity. Allison deserves a big
  round of thanks for her ongoing work to get this functionality into
  XFS.

  There are also many other smaller changes and improvements, so overall
  this is one of the bigger XFS merge requests in some time.

  I will be following up next week with another smaller pull request -
  we already have another round of fixes and improvements to the logged
  attribute replay functionality just about ready to go. They'll soak
  and test over the next week, and I'll send a pull request for them
  near the end of the merge window.

  Summary:

   - support for printk message indexing.

   - large extent counts to provide support for up to 2^47 data extents
     and 2^32 attribute extents, allowing us to scale beyond 4 billion
     data extents to billions of xattrs per inode.

   - conversion of various flags fields to be consistently declared as
     unsigned bit fields.

   - improvements to realtime extent accounting and converts them to
     per-cpu counters to match all the other block and inode accounting.

   - reworks core log formatting code to reduce iterations, have a
     shorter, cleaner fast path and generally be easier to understand
     and maintain.

   - improvements to rmap btree searches that reduce overhead by up to
     30% resulting in xfs_scrub runtime reductions of 15%.

   - improvements to reflink that remove the size limitations in
     remapping operations and greatly reduce the size of transaction
     reservations.

   - reworks the minimum log size calculations to allow us to change
     transaction reservations without changing the minimum supported log
     size.

   - removal of quota warning support as it has never been used on
     Linux.

   - intent whiteouts to allow us to cancel intents that are completed
     entirely in memory rather than having use CPU and disk bandwidth
     formatting and writing them into the journal when it is not
     necessary. This makes rmap, reflink and extent freeing slightly
     more efficient, but provides massive improvements for....

   - Logged Attribute Replay feature support. This is a fundamental
     change to the way we modify attributes, laying the foundation for
     future integration of attribute modifications as part of other
     atomic transactional operations the filesystem performs.

   - Lots of cleanups and fixes for the logged attribute replay
     functionality"

* tag 'xfs-5.19-for-linus' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (124 commits)
  xfs: can't use kmem_zalloc() for attribute buffers
  xfs: detect empty attr leaf blocks in xfs_attr3_leaf_verify
  xfs: ATTR_REPLACE algorithm with LARP enabled needs rework
  xfs: use XFS_DA_OP flags in deferred attr ops
  xfs: remove xfs_attri_remove_iter
  xfs: switch attr remove to xfs_attri_set_iter
  xfs: introduce attr remove initial states into xfs_attr_set_iter
  xfs: xfs_attr_set_iter() does not need to return EAGAIN
  xfs: clean up final attr removal in xfs_attr_set_iter
  xfs: remote xattr removal in xfs_attr_set_iter() is conditional
  xfs: XFS_DAS_LEAF_REPLACE state only needed if !LARP
  xfs: split remote attr setting out from replace path
  xfs: consolidate leaf/node states in xfs_attr_set_iter
  xfs: kill XFS_DAC_LEAF_ADDNAME_INIT
  xfs: separate out initial attr_set states
  xfs: don't set quota warning values
  xfs: remove warning counters from struct xfs_dquot_res
  xfs: remove quota warning limit from struct xfs_quota_limits
  xfs: rework deferred attribute operation setup
  xfs: make xattri_leaf_bp more useful
  ...
parents e375780b efd409a4

fs/xfs/Makefile

+1 −0
Original line number Diff line number Diff line
@@ -102,6 +102,7 @@ xfs-y += xfs_log.o \ 
				   xfs_buf_item_recover.o \

				   xfs_dquot_item_recover.o \

				   xfs_extfree_item.o \

				   xfs_attr_item.o \

				   xfs_icreate_item.o \

				   xfs_inode_item.o \

				   xfs_inode_item_recover.o \

fs/xfs/libxfs/xfs_alloc.c

+6 −6
Original line number Diff line number Diff line
@@ -2511,7 +2511,7 @@ __xfs_free_extent_later( 


	ASSERT(bno != NULLFSBLOCK);

	ASSERT(len > 0);

	ASSERT(len <= MAXEXTLEN);

	ASSERT(len <= XFS_MAX_BMBT_EXTLEN);

	ASSERT(!isnullstartblock(bno));

	agno = XFS_FSB_TO_AGNO(mp, bno);

	agbno = XFS_FSB_TO_AGBNO(mp, bno);
@@ -2777,7 +2777,7 @@ xfs_alloc_get_freelist( 
	xfs_agblock_t		bno;

	__be32			*agfl_bno;

	int			error;

	int			logflags;

	uint32_t		logflags;

	struct xfs_mount	*mp = tp->t_mountp;

	struct xfs_perag	*pag;
@@ -2830,9 +2830,9 @@ xfs_alloc_get_freelist( 
 */

void

xfs_alloc_log_agf(

	xfs_trans_t	*tp,	/* transaction pointer */

	struct xfs_buf	*bp,	/* buffer for a.g. freelist header */

	int		fields)	/* mask of fields to be logged (XFS_AGF_...) */

	struct xfs_trans	*tp,

	struct xfs_buf		*bp,

	uint32_t		fields)

{

	int	first;		/* first byte offset */

	int	last;		/* last byte offset */
@@ -2902,7 +2902,7 @@ xfs_alloc_put_freelist( 
	struct xfs_perag	*pag;

	__be32			*blockp;

	int			error;

	int			logflags;

	uint32_t		logflags;

	__be32			*agfl_bno;

	int			startoff;

fs/xfs/libxfs/xfs_alloc.h

+1 −1
Original line number Diff line number Diff line
@@ -121,7 +121,7 @@ void 
xfs_alloc_log_agf(

	struct xfs_trans *tp,	/* transaction pointer */

	struct xfs_buf	*bp,	/* buffer for a.g. freelist header */

	int		fields);/* mask of fields to be logged (XFS_AGF_...) */

	uint32_t	fields);/* mask of fields to be logged (XFS_AGF_...) */



/*

 * Interface for inode allocation to force the pag data to be initialized.

fs/xfs/libxfs/xfs_attr.c

+801 −843

File changed.

Preview size limit exceeded, changes collapsed.

fs/xfs/libxfs/xfs_attr.h

+168 −30
Original line number Diff line number Diff line
@@ -28,6 +28,15 @@ struct xfs_attr_list_context; 
 */

#define	ATTR_MAX_VALUELEN	(64*1024)	/* max length of a value */



static inline bool xfs_has_larp(struct xfs_mount *mp)

{

#ifdef DEBUG

	return xfs_globals.larp;

#else

	return false;

#endif

}



/*

 * Kernel-internal version of the attrlist cursor.

 */
@@ -425,7 +434,7 @@ struct xfs_attr_list_context { 
 */



/*

 * Enum values for xfs_delattr_context.da_state

 * Enum values for xfs_attr_item.xattri_da_state

 *

 * These values are used by delayed attribute operations to keep track  of where

 * they were before they returned -EAGAIN.  A return code of -EAGAIN signals the
@@ -435,45 +444,104 @@ struct xfs_attr_list_context { 
 */

enum xfs_delattr_state {

	XFS_DAS_UNINIT		= 0,	/* No state has been set yet */

	XFS_DAS_RMTBLK,		      /* Removing remote blks */

	XFS_DAS_RM_NAME,	      /* Remove attr name */

	XFS_DAS_RM_SHRINK,	      /* We are shrinking the tree */

	XFS_DAS_FOUND_LBLK,	      /* We found leaf blk for attr */

	XFS_DAS_FOUND_NBLK,	      /* We found node blk for attr */

	XFS_DAS_FLIP_LFLAG,	      /* Flipped leaf INCOMPLETE attr flag */

	XFS_DAS_RM_LBLK,	      /* A rename is removing leaf blocks */

	XFS_DAS_RD_LEAF,	      /* Read in the new leaf */

	XFS_DAS_ALLOC_NODE,	      /* We are allocating node blocks */

	XFS_DAS_FLIP_NFLAG,	      /* Flipped node INCOMPLETE attr flag */

	XFS_DAS_RM_NBLK,	      /* A rename is removing node blocks */

	XFS_DAS_CLR_FLAG,	      /* Clear incomplete flag */



	/*

	 * Initial sequence states. The replace setup code relies on the

	 * ADD and REMOVE states for a specific format to be sequential so

	 * that we can transform the initial operation to be performed

	 * according to the xfs_has_larp() state easily.

	 */

	XFS_DAS_SF_ADD,			/* Initial sf add state */

	XFS_DAS_SF_REMOVE,		/* Initial sf replace/remove state */



	XFS_DAS_LEAF_ADD,		/* Initial leaf add state */

	XFS_DAS_LEAF_REMOVE,		/* Initial leaf replace/remove state */



	XFS_DAS_NODE_ADD,		/* Initial node add state */

	XFS_DAS_NODE_REMOVE,		/* Initial node replace/remove state */



	/* Leaf state set/replace/remove sequence */

	XFS_DAS_LEAF_SET_RMT,		/* set a remote xattr from a leaf */

	XFS_DAS_LEAF_ALLOC_RMT,		/* We are allocating remote blocks */

	XFS_DAS_LEAF_REPLACE,		/* Perform replace ops on a leaf */

	XFS_DAS_LEAF_REMOVE_OLD,	/* Start removing old attr from leaf */

	XFS_DAS_LEAF_REMOVE_RMT,	/* A rename is removing remote blocks */

	XFS_DAS_LEAF_REMOVE_ATTR,	/* Remove the old attr from a leaf */



	/* Node state sequence, must match leaf state above */

	XFS_DAS_NODE_SET_RMT,		/* set a remote xattr from a node */

	XFS_DAS_NODE_ALLOC_RMT,		/* We are allocating remote blocks */

	XFS_DAS_NODE_REPLACE,		/* Perform replace ops on a node */

	XFS_DAS_NODE_REMOVE_OLD,	/* Start removing old attr from node */

	XFS_DAS_NODE_REMOVE_RMT,	/* A rename is removing remote blocks */

	XFS_DAS_NODE_REMOVE_ATTR,	/* Remove the old attr from a node */



	XFS_DAS_DONE,			/* finished operation */

};



#define XFS_DAS_STRINGS	\

	{ XFS_DAS_UNINIT,		"XFS_DAS_UNINIT" }, \

	{ XFS_DAS_SF_ADD,		"XFS_DAS_SF_ADD" }, \

	{ XFS_DAS_SF_REMOVE,		"XFS_DAS_SF_REMOVE" }, \

	{ XFS_DAS_LEAF_ADD,		"XFS_DAS_LEAF_ADD" }, \

	{ XFS_DAS_LEAF_REMOVE,		"XFS_DAS_LEAF_REMOVE" }, \

	{ XFS_DAS_NODE_ADD,		"XFS_DAS_NODE_ADD" }, \

	{ XFS_DAS_NODE_REMOVE,		"XFS_DAS_NODE_REMOVE" }, \

	{ XFS_DAS_LEAF_SET_RMT,		"XFS_DAS_LEAF_SET_RMT" }, \

	{ XFS_DAS_LEAF_ALLOC_RMT,	"XFS_DAS_LEAF_ALLOC_RMT" }, \

	{ XFS_DAS_LEAF_REPLACE,		"XFS_DAS_LEAF_REPLACE" }, \

	{ XFS_DAS_LEAF_REMOVE_OLD,	"XFS_DAS_LEAF_REMOVE_OLD" }, \

	{ XFS_DAS_LEAF_REMOVE_RMT,	"XFS_DAS_LEAF_REMOVE_RMT" }, \

	{ XFS_DAS_LEAF_REMOVE_ATTR,	"XFS_DAS_LEAF_REMOVE_ATTR" }, \

	{ XFS_DAS_NODE_SET_RMT,		"XFS_DAS_NODE_SET_RMT" }, \

	{ XFS_DAS_NODE_ALLOC_RMT,	"XFS_DAS_NODE_ALLOC_RMT" },  \

	{ XFS_DAS_NODE_REPLACE,		"XFS_DAS_NODE_REPLACE" },  \

	{ XFS_DAS_NODE_REMOVE_OLD,	"XFS_DAS_NODE_REMOVE_OLD" }, \

	{ XFS_DAS_NODE_REMOVE_RMT,	"XFS_DAS_NODE_REMOVE_RMT" }, \

	{ XFS_DAS_NODE_REMOVE_ATTR,	"XFS_DAS_NODE_REMOVE_ATTR" }, \

	{ XFS_DAS_DONE,			"XFS_DAS_DONE" }



/*

 * Defines for xfs_delattr_context.flags

 * Defines for xfs_attr_item.xattri_flags

 */

#define XFS_DAC_DEFER_FINISH		0x01 /* finish the transaction */

#define XFS_DAC_LEAF_ADDNAME_INIT	0x02 /* xfs_attr_leaf_addname init*/

#define XFS_DAC_LEAF_ADDNAME_INIT	0x01 /* xfs_attr_leaf_addname init*/



/*

 * Context used for keeping track of delayed attribute operations

 */

struct xfs_delattr_context {

	struct xfs_da_args      *da_args;

struct xfs_attr_item {

	struct xfs_da_args		*xattri_da_args;



	/*

	 * Used by xfs_attr_set to hold a leaf buffer across a transaction roll

	 */

	struct xfs_buf			*xattri_leaf_bp;



	/* Used in xfs_attr_rmtval_set_blk to roll through allocating blocks */

	struct xfs_bmbt_irec	map;

	xfs_dablk_t		lblkno;

	int			blkcnt;

	struct xfs_bmbt_irec		xattri_map;

	xfs_dablk_t			xattri_lblkno;

	int				xattri_blkcnt;



	/* Used in xfs_attr_node_removename to roll through removing blocks */

	struct xfs_da_state     *da_state;

	struct xfs_da_state		*xattri_da_state;



	/* Used to keep track of current state of delayed operation */

	unsigned int            flags;

	enum xfs_delattr_state  dela_state;

	unsigned int			xattri_flags;

	enum xfs_delattr_state		xattri_dela_state;



	/*

	 * Attr operation being performed - XFS_ATTR_OP_FLAGS_*

	 */

	unsigned int			xattri_op_flags;



	/*

	 * used to log this item to an intent containing a list of attrs to

	 * commit later

	 */

	struct list_head		xattri_list;

};





/*========================================================================

 * Function prototypes for the kernel.

 *========================================================================*/
@@ -489,11 +557,81 @@ bool xfs_attr_is_leaf(struct xfs_inode *ip); 
int xfs_attr_get_ilocked(struct xfs_da_args *args);

int xfs_attr_get(struct xfs_da_args *args);

int xfs_attr_set(struct xfs_da_args *args);

int xfs_attr_set_args(struct xfs_da_args *args);

int xfs_attr_remove_args(struct xfs_da_args *args);

int xfs_attr_remove_iter(struct xfs_delattr_context *dac);

int xfs_attr_set_iter(struct xfs_attr_item *attr);

int xfs_attr_remove_iter(struct xfs_attr_item *attr);

bool xfs_attr_namecheck(const void *name, size_t length);

void xfs_delattr_context_init(struct xfs_delattr_context *dac,

			      struct xfs_da_args *args);

int xfs_attr_calc_size(struct xfs_da_args *args, int *local);

void xfs_init_attr_trans(struct xfs_da_args *args, struct xfs_trans_res *tres,

			 unsigned int *total);



extern struct kmem_cache	*xfs_attri_cache;

extern struct kmem_cache	*xfs_attrd_cache;



int __init xfs_attri_init_cache(void);

void xfs_attri_destroy_cache(void);

int __init xfs_attrd_init_cache(void);

void xfs_attrd_destroy_cache(void);



/*

 * Check to see if the attr should be upgraded from non-existent or shortform to

 * single-leaf-block attribute list.

 */

static inline bool

xfs_attr_is_shortform(

	struct xfs_inode    *ip)

{

	return ip->i_afp->if_format == XFS_DINODE_FMT_LOCAL ||

	       (ip->i_afp->if_format == XFS_DINODE_FMT_EXTENTS &&

		ip->i_afp->if_nextents == 0);

}



static inline enum xfs_delattr_state

xfs_attr_init_add_state(struct xfs_da_args *args)

{

	/*

	 * When called from the completion of a attr remove to determine the

	 * next state, the attribute fork may be null. This can occur only occur

	 * on a pure remove, but we grab the next state before we check if a

	 * replace operation is being performed. If we are called from any other

	 * context, i_afp is guaranteed to exist. Hence if the attr fork is

	 * null, we were called from a pure remove operation and so we are done.

	 */

	if (!args->dp->i_afp)

		return XFS_DAS_DONE;



	args->op_flags |= XFS_DA_OP_ADDNAME;

	if (xfs_attr_is_shortform(args->dp))

		return XFS_DAS_SF_ADD;

	if (xfs_attr_is_leaf(args->dp))

		return XFS_DAS_LEAF_ADD;

	return XFS_DAS_NODE_ADD;

}



static inline enum xfs_delattr_state

xfs_attr_init_remove_state(struct xfs_da_args *args)

{

	args->op_flags |= XFS_DA_OP_REMOVE;

	if (xfs_attr_is_shortform(args->dp))

		return XFS_DAS_SF_REMOVE;

	if (xfs_attr_is_leaf(args->dp))

		return XFS_DAS_LEAF_REMOVE;

	return XFS_DAS_NODE_REMOVE;

}



/*

 * If we are logging the attributes, then we have to start with removal of the

 * old attribute so that there is always consistent state that we can recover

 * from if the system goes down part way through. We always log the new attr

 * value, so even when we remove the attr first we still have the information in

 * the log to finish the replace operation atomically.

 */

static inline enum xfs_delattr_state

xfs_attr_init_replace_state(struct xfs_da_args *args)

{

	args->op_flags |= XFS_DA_OP_ADDNAME | XFS_DA_OP_REPLACE;

	if (xfs_has_larp(args->dp->i_mount))

		return xfs_attr_init_remove_state(args);

	return xfs_attr_init_add_state(args);

}



#endif	/* __XFS_ATTR_H__ */