Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
9d4482ec
Commit
9d4482ec
authored
Jun 23, 2020
by
George Koltsov
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Use #in_batches in Project Export StreamingSerializer
parent
1232af43
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
24 additions
and
2 deletions
+24
-2
lib/gitlab/import_export/json/streaming_serializer.rb
lib/gitlab/import_export/json/streaming_serializer.rb
+10
-2
spec/lib/gitlab/import_export/json/streaming_serializer_spec.rb
...ib/gitlab/import_export/json/streaming_serializer_spec.rb
+14
-0
No files found.
lib/gitlab/import_export/json/streaming_serializer.rb
View file @
9d4482ec
...
...
@@ -69,8 +69,16 @@ module Gitlab
key_preloads
=
preloads
&
.
dig
(
key
)
records
=
records
.
preload
(
key_preloads
)
if
key_preloads
records
.
find_each
(
batch_size:
batch_size
)
do
|
record
|
items
<<
Raw
.
new
(
record
.
to_json
(
options
))
records
.
in_batches
(
of:
batch_size
)
do
|
batch
|
# rubocop:disable Cop/InBatches
# order each batch by its primary key to ensure
# consistent and predictable ordering of each exported relation
# as additional `WHERE` clauses can impact the order in which data is being
# returned by database when no `ORDER` is specified
batch
=
batch
.
reorder
(
batch
.
klass
.
primary_key
)
batch
.
each
do
|
record
|
items
<<
Raw
.
new
(
record
.
to_json
(
options
))
end
end
end
...
...
spec/lib/gitlab/import_export/json/streaming_serializer_spec.rb
View file @
9d4482ec
...
...
@@ -61,6 +61,20 @@ describe Gitlab::ImportExport::JSON::StreamingSerializer do
subject
.
execute
end
context
'relation ordering'
do
before
do
create_list
(
:issue
,
5
,
project:
exportable
)
end
it
'orders exported issues by primary key'
do
expected_issues
=
exportable
.
issues
.
reorder
(
:id
).
map
(
&
:to_json
)
expect
(
json_writer
).
to
receive
(
:write_relation_array
).
with
(
exportable_path
,
:issues
,
expected_issues
)
subject
.
execute
end
end
end
context
'with single relation'
do
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment