Commits · b4e6f66e45b43aed0903731b6c0700573f88282a · Kirill Smelkov / linux

03 Oct, 2022 40 commits

ksm: use a folio in replace_page() · b4e6f66e

Matthew Wilcox (Oracle) authored Sep 02, 2022

Replace three calls to compound_head() with one.

Link: https://lkml.kernel.org/r/20220902194653.1739778-46-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

b4e6f66e

uprobes: use folios more widely in __replace_page() · 5fcd079a

Matthew Wilcox (Oracle) authored Sep 02, 2022

Remove a few hidden calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-45-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

5fcd079a

madvise: convert madvise_free_pte_range() to use a folio · 98b211d6

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves a lot of calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-44-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

98b211d6

huge_memory: convert do_huge_pmd_wp_page() to use a folio · 2fad3d14

Matthew Wilcox (Oracle) authored Sep 02, 2022

Removes many calls to compound_head(). Does not remove the assumption
that a folio may not be larger than a PMD.

Link: https://lkml.kernel.org/r/20220902194653.1739778-43-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

2fad3d14

mm: convert do_wp_page() to use a folio · e4a2ed94

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves many calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-42-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

e4a2ed94

swap: convert swap_writepage() to use a folio · 71fa1a53

Matthew Wilcox (Oracle) authored Sep 02, 2022

Removes many calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-41-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

71fa1a53

swap_state: convert free_swap_cache() to use a folio · aedd74d4

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves several calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-40-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

aedd74d4

mm: remove lookup_swap_cache() · cb691e2f

Matthew Wilcox (Oracle) authored Sep 02, 2022

All callers have now been converted to swap_cache_get_folio(), so we can
remove this wrapper.

Link: https://lkml.kernel.org/r/20220902194653.1739778-39-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

cb691e2f

mm: convert do_swap_page() to use swap_cache_get_folio() · 5a423081

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves a folio->page->folio conversion.

Link: https://lkml.kernel.org/r/20220902194653.1739778-38-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

5a423081

swapfile: convert unuse_pte_range() to use a folio · f102cd8b

Matthew Wilcox (Oracle) authored Sep 02, 2022

Delay fetching the precise page from the folio until we're in unuse_pte().
Saves many calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-37-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

f102cd8b

swapfile: convert __try_to_reclaim_swap() to use a folio · 2c3f6194

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves five calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-36-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

2c3f6194

swapfile: convert try_to_unuse() to use a folio · 000085b9

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves five calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-35-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

000085b9

shmem: remove shmem_getpage() · 923e2f0e

Matthew Wilcox (Oracle) authored Sep 02, 2022

With all callers removed, remove this wrapper function. The flags are now
mysteriously called SGP, but I think we can live with that.

Link: https://lkml.kernel.org/r/20220902194653.1739778-34-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

923e2f0e

userfaultfd: convert mcontinue_atomic_pte() to use a folio · 12acf4fb

Matthew Wilcox (Oracle) authored Sep 02, 2022

shmem_getpage() is being replaced by shmem_get_folio() so use a folio
throughout this function. Saves several calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-33-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

12acf4fb

khugepaged: call shmem_get_folio() · 7459c149

Matthew Wilcox (Oracle) authored Sep 02, 2022

shmem_getpage() is being removed, so call its replacement and find the
precise page ourselves.

Link: https://lkml.kernel.org/r/20220902194653.1739778-32-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

7459c149

shmem: convert shmem_get_link() to use a folio · e4b57722

Matthew Wilcox (Oracle) authored Sep 02, 2022

Symlinks will never use a large folio, but using the folio API removes a
lot of unnecessary folio->page->folio conversions.

Link: https://lkml.kernel.org/r/20220902194653.1739778-31-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

e4b57722

shmem: convert shmem_symlink() to use a folio · 7ad0414b

Matthew Wilcox (Oracle) authored Sep 02, 2022

While symlinks will always be < PAGE_SIZE, using the folio APIs gets rid
of unnecessary calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-30-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

7ad0414b

shmem: convert shmem_fallocate() to use a folio · b0802b22

Matthew Wilcox (Oracle) authored Sep 02, 2022

Call shmem_get_folio() and use the folio APIs instead of the page APIs.
Saves several calls to compound_head() and removes assumptions about the
size of a large folio.

Link: https://lkml.kernel.org/r/20220902194653.1739778-29-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

b0802b22

shmem: convert shmem_file_read_iter() to use shmem_get_folio() · 4601e2fc

Matthew Wilcox (Oracle) authored Sep 02, 2022

Use a folio throughout, saving five calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-28-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

4601e2fc

shmem: convert shmem_write_begin() to use shmem_get_folio() · eff1f906

Matthew Wilcox (Oracle) authored Sep 02, 2022

Use a folio throughout this function, saving a couple of calls to
compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-27-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

eff1f906

shmem: convert shmem_get_partial_folio() to use shmem_get_folio() · a7f5862c

Matthew Wilcox (Oracle) authored Sep 02, 2022

Get rid of an unnecessary folio->page->folio conversion.

Link: https://lkml.kernel.org/r/20220902194653.1739778-26-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

a7f5862c

shmem: add shmem_get_folio() · 4e1fc793

Matthew Wilcox (Oracle) authored Sep 02, 2022

With no remaining callers of shmem_getpage_gfp(), add shmem_get_folio()
and reimplement shmem_getpage() as a call to shmem_get_folio().

Link: https://lkml.kernel.org/r/20220902194653.1739778-25-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

4e1fc793

shmem: convert shmem_read_mapping_page_gfp() to use shmem_get_folio_gfp() · a3a9c397

Matthew Wilcox (Oracle) authored Sep 02, 2022

Saves a couple of calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-24-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

a3a9c397

shmem: convert shmem_fault() to use shmem_get_folio_gfp() · 68a54100

Matthew Wilcox (Oracle) authored Sep 02, 2022

No particular advantage for this function, but necessary to remove
shmem_getpage_gfp().

[hughd@google.com: fix crash]
  Link: https://lkml.kernel.org/r/7693a84-bdc2-27b5-2695-d0fe8566571f@google.com
Link: https://lkml.kernel.org/r/20220902194653.1739778-23-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

68a54100

shmem: convert shmem_getpage_gfp() to shmem_get_folio_gfp() · fc26babb

Matthew Wilcox (Oracle) authored Sep 02, 2022

Add a shmem_getpage_gfp() wrapper for compatibility with current users.

Link: https://lkml.kernel.org/r/20220902194653.1739778-22-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

fc26babb

shmem: eliminate struct page from shmem_swapin_folio() · 5739a81c

Matthew Wilcox (Oracle) authored Sep 02, 2022

Convert shmem_swapin() to return a folio and use swap_cache_get_folio(),
removing all uses of struct page in this function.

Link: https://lkml.kernel.org/r/20220902194653.1739778-21-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

5739a81c

swap: add swap_cache_get_folio() · c9edc242

Matthew Wilcox (Oracle) authored Sep 02, 2022

Convert lookup_swap_cache() into swap_cache_get_folio() and add a
lookup_swap_cache() wrapper around it.

[akpm@linux-foundation.org: add CONFIG_SWAP=n stub for swap_cache_get_folio()]
Link: https://lkml.kernel.org/r/20220902194653.1739778-20-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

c9edc242

shmem: convert shmem_replace_page() to shmem_replace_folio() · 0d698e25

Matthew Wilcox (Oracle) authored Sep 02, 2022

The caller has a folio, so convert the calling convention and rename the
function.

Link: https://lkml.kernel.org/r/20220902194653.1739778-19-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

0d698e25

shmem: convert shmem_mfill_atomic_pte() to use a folio · 7a7256d5

Matthew Wilcox (Oracle) authored Sep 02, 2022

Assert that this is a single-page folio as there are several assumptions
in here that it's exactly PAGE_SIZE bytes large. Saves several calls to
compound_head() and removes the last caller of shmem_alloc_page().

Link: https://lkml.kernel.org/r/20220902194653.1739778-18-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

7a7256d5

memcg: convert mem_cgroup_swapin_charge_page() to mem_cgroup_swapin_charge_folio() · 65995918

Matthew Wilcox (Oracle) authored Sep 02, 2022

All callers now have a folio, so pass it in here and remove an unnecessary
call to page_folio().

Link: https://lkml.kernel.org/r/20220902194653.1739778-17-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

65995918

mm: convert do_swap_page()'s swapcache variable to a folio · d4f9565a

Matthew Wilcox (Oracle) authored Sep 02, 2022

The 'swapcache' variable is used to track whether the page is from the
swapcache or not. It can do this equally well by being the folio of the
page rather than the page itself, and this saves a number of calls to
compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-16-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

d4f9565a

mm: convert do_swap_page() to use a folio · 63ad4add

Matthew Wilcox (Oracle) authored Sep 02, 2022

Removes quite a lot of calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-15-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

63ad4add

mm/swap: convert put_swap_page() to put_swap_folio() · 4081f744

Matthew Wilcox (Oracle) authored Sep 02, 2022

With all callers now using a folio, we can convert this function.

Link: https://lkml.kernel.org/r/20220902194653.1739778-14-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

4081f744

mm/swap: convert add_to_swap_cache() to take a folio · a4c366f0

Matthew Wilcox (Oracle) authored Sep 02, 2022

With all callers using folios, we can convert add_to_swap_cache() to take
a folio and use it throughout.

Link: https://lkml.kernel.org/r/20220902194653.1739778-13-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

a4c366f0

mm/swap: convert __read_swap_cache_async() to use a folio · a0d3374b

Matthew Wilcox (Oracle) authored Sep 02, 2022

Remove a few hidden (and one visible) calls to compound_head().

Link: https://lkml.kernel.org/r/20220902194653.1739778-12-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

a0d3374b

mm/swapfile: convert try_to_free_swap() to folio_free_swap() · bdb0ed54

Matthew Wilcox (Oracle) authored Sep 02, 2022

Add kernel-doc for folio_free_swap() and make it return bool. Add a
try_to_free_swap() compatibility wrapper.

Link: https://lkml.kernel.org/r/20220902194653.1739778-11-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

bdb0ed54

mm/swapfile: remove page_swapcount() · 14d01ee9

Matthew Wilcox (Oracle) authored Sep 02, 2022

By restructuring folio_swapped(), it can use swap_swapcount() instead of
page_swapcount(). It's even a little more efficient.

Link: https://lkml.kernel.org/r/20220902194653.1739778-10-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

14d01ee9

shmem: convert shmem_replace_page() to use folios throughout · 907ea17e

Matthew Wilcox (Oracle) authored Sep 02, 2022

Introduce folio_set_swap_entry() to abstract how both folio->private and
swp_entry_t work. Use swap_address_space() directly instead of
indirecting through folio_mapping(). Include an assertion that the old
folio is not large as we only allocate a single-page folio to replace it.
Use folio_put_refs() instead of calling folio_put() twice.

Link: https://lkml.kernel.org/r/20220902194653.1739778-9-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

907ea17e

shmem: convert shmem_delete_from_page_cache() to take a folio · 4cd400fd

Matthew Wilcox (Oracle) authored Sep 02, 2022

Remove the assertion that the page is not Compound as this function now
handles large folios correctly.

Link: https://lkml.kernel.org/r/20220902194653.1739778-8-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

4cd400fd

shmem: convert shmem_writepage() to use a folio throughout · f530ed0e

Matthew Wilcox (Oracle) authored Sep 02, 2022

Even though we will split any large folio that comes in, write the code to
handle large folios so as to not leave a trap for whoever tries to handle
large folios in the swap cache.

Link: https://lkml.kernel.org/r/20220902194653.1739778-7-willy@infradead.orgSigned-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

f530ed0e