GitLab

Change-Id: Ie70b72005a6d3df47adb0e5b1b21b9d7b99b8ab0

This is done by sorting inputs before processing them.

Bug: 23305646

Change-Id: Ia55c481f4eb342bc925bc100299324ea470bf9a0

This version includes "Support --num-threads with --multi-dex"

(cherry picked from commit 7d97556f)

Change-Id: I712145094205d56358ef0b9ff51f137907d972e9

Revert change to dex rotation algorithm made as part of commit:
845d9d0e

Bug: https://code.google.com/p/android/issues/detail?id=161887

Signed-off-by: Peter Jensen <jensenp@google.com>

(cherry picked from commit d024c067)

Change-Id: Iede742620854cef3542bf98b77e66aa8a305195e

  With fix for regression introduced in original commit.

The current dx implementation supports options --multi-dex, for applications
not fitting within the dex format limitations; and --num-threads=N, triggers
concurrent processing of multiple input files. However, the implementation
has the following limitations:

The --num-threads option is disabled when used together with --multi-dex.
The --num-threads option implements concurrency at the level of classpath
entries, and does nothing when the classes to be translated are specified
with a single classpath element (e.g. single jar output from Proguard).
The existing --num-threads implementation may produce indeterministic output.
The heuristic used by the --multi-dex option to determine when to rotate the
dex output file is overly conservative.

The primary objective of this change is:
Concurrent translation of classes, independently of input specification format.
Support --num-threads=N in both mono- and multi-dex mode.
Deterministic class output order.
Near optimal use of dex file format capacity.

This is accomplished by reorganizing the dx workflow in a pipeline of
concurrent phases.

read-class  | parse-class | translate-class | add-to-dex | convert-dex-to-byte[];
output-dex-files-or-jar

To manage dex file rotation (i.e. --multi-dex support), the parse-class and
add-to-dex phases are synchronized to prevent forwarding classes to the
translate-class phase if it could potentially result in breaking the dex
format limitations. The heuristic currently used to estimate the number of
indices needed for a class is improved, to minimize the amount of serialization
imposed by this feedback mechanism, and to improve the use of dex file capacity.

The translate-class and convert-dex-to-byte[] phases are further parallelized
with configurable (--num-threads=N option) thread pools. This allow translating
classes concurrently, while also performing output conversion in parallel.
Separate collector threads are used to collect results from the thread pools
in deterministic order.

Testing was performed on an Ubuntu system, with 6 cores and 12 hardware threads.
The taskset command was used to experimentally establish that running with more
than 8 hardware threads does not provide any additional benefit.

Experiments shows that the argument to --num-threads should not exceed the
lesser of the number of available hardware threads, and 5. Setting it to a
higher value results in no additional benefit.

The gain is generally larger for larger applications, and not significant for
small applications with less than a few thousands classes. Experiments with
generated classes shows that for large applications gains as high as 50% may
be possible.

For an existing real-life application with more than 11k classes, and requiring
2 dex files, a speed-up of 37% was achieved (--num-threads=5, 8 hardware
threads, 4g Java heap). A speedup of 31% was observed for another application
with ~7 classes.

For small applications, use of --num-threads=N>1 doesn’t provide significant
benefit. Running with --num-threads=1, the modified dx is slightly faster,
but no significant gain is observed unless the application requires multiple
dex files.

The one case where a significant regression may be observed is when using
--num-threads=N>1, with a single hardware thread. This is an inappropriate
configuration, even with the current implementation. However, because of
the limitations of the current implementation, such configurations may exist.
For instance, a configuration using both --multi-dex and --num-threads=5 will
currently generate a warning about using the two options together. With the
new implementation, the options can legitimately be used together, and could
result in an ~20% regression running on a single hardware thread.
Note: the current dx implementation, without --num-threads option, is already
approximately 50% slower  with 1 hardware thread, compared to running with 2
or more. With 2 hardware threads the implementations are practically at par
(a little better, or a little worse, depending on the application).

Testing:
Tested with 6 existing applications ranging in size from 1K - 12K classes.
Updated and tested with relevant existing unit tests (one test changed to
account for better dex rotation heuristic).
Added unit test to test deterministic output.
Added unit performance test. By default run script merely validates that
--multi-dex and --num-threads can be used together (fast). However, the test
is configurable to perform performance test, over sets of generated classes.
Signed-off-by: Peter Jensen <jensenp@google.com>

(cherry picked from commit 845d9d0e)

(cherry picked from commit dd140a22)

Change-Id: I33a8ea0451efc0af7eb1d72e80cb926d6583d569

This is a structural change, and breaks code that looks up
directory resource names (icu4j for example).

This change also includes a minor cosmetic change to use a
while() loop instead of for(;;).

bug: 19108324

(cherry picked from commit 7736e8ff)

Change-Id: Ifcbc60cc7808334cdfdd4f27bf137a8f6455117d

Bug: 18486731

(cherry picked from commit bb404aa9)

Change-Id: I048f6064cc49c78df21a1c42316761502a0a5366

This commit adds a new command line option to dx:
--no-warning.
If the caller appends this to the arguments, dx won't print
warnings to System.err. This can be useful if the user
dexes external jars with old class format, so dx would
print lot of warning about those.

Bug: https://code.google.com/p/android/issues/detail?id=78285

Signed-off-by: Csaba Kozák <kozakcsabi@gmail.com>

(cherry picked from commit ef1de423)

Change-Id: I6d09c684b5eb97aa28e0b12e3ef44b46293c7dec

We have to parse descriptors and treat the types they reference as direct
reference as we do for other direct type references. This is required because
Dalvik verifier is checking those descriptor the same way as it does for other
type references. See:
https://android.googlesource.com/platform/dalvik/+/kitkat-mr2.2-release/vm/analysis/CodeVerify.cpp#637

Bug: 19626444

(cherry picked from commit 8f9fb34f)

Change-Id: Ib9f43dd0c5f0b983fcc80e13caa13fd8b2dc2244

We force all classes annotated with runtime annotation to be kept
in the main dex list. This allow to workaround a Dalvik bug when
accessing the annotations. But this increase the pressure in the
main dex indexes. The new option is to allow to disable this
workaround for application facing the index limit in their main
dex and that don't access annotation at runtime or that have only
annotations not subject to the resolution bug.

Bug: 18928046
Change-Id: Ibcd7e579a7fef3451ec8aeb266ea67514d82cd50

* commit '8e88bab0':
  Keep annotated classes in main dex list

Only classes annotated with a runtime visible annotation are to be kept.
See https://code.google.com/p/android/issues/detail?id=78144

Bug: 18385117

(cherry picked from commit 893795fc)

Change-Id: I46ceef4aab5119ba4515a07e8f17577e15931dbd

Dumps from Android devices have multiple heaps, including Zygote,
image, etc.  When investigating RAM usage, it's most useful to focus
on "real" allocations from the app, ignoring the other heaps.

This change introduces the "-z" flag which will drop allocations
on non-app heaps during conversion.

Bug: 18473132
Change-Id: Ib1d5256b7bc4cfd7791f00c18a1cff920e0e3f46

Merging dex files starting with smaller ones can have
a big impact on performance as it potentially reduces
the number of compact events during the merge.

Gradle will now send the list sorted in ascending order
of file size, and dex should keep this order when parsing
the input list file.

Change-Id: Iff3ae4d30106fe70dca0764b33addfecceb84d09

Change-Id: Iaea2a8ec163cdfc50a5647e4071735675eee3481

This new version should include the following fixes:
8863d0fd (Bug: 16709921)
0ab4f7a318db3635c870ac50b669085312feb567 (Bug: 17438982)

(cherry picked from commit 70df3df6)

Change-Id: I986804c02ba807dac5bbfc9a280e0c0f5a684419

On Windows machines the command line can only accept
commands that are 8192 characters or less.  This can
cause a problem when a project has many dependencies
and many libraries are being passed to dex at once.

This fixes it by adding a new option to dex called
--input-list that can read inputs from a file.
This way we only have to pass a single file name
instead of a ton of jar files. This keeps the command
short so Windows will not complain about it.

Bug: 17438982
Signed-off-by: John Fazekas <jfazekas@fd-software.com>

(cherry picked from commit 914f2263)

Change-Id: I6edd1e7c1d1fddc8d6bf4c4963295d4f4e64032e

This is needed since class constants may not be unique. Indeed, there
may be multiple identical class contants in the constant pool.
Bug: 16709921

(cherry picked from commit 8863d0fd)

Change-Id: I63c07ebf03d67eb942aec6364c7e02c631744612

This fixed bogus docs build errors like:
dalvik/dx/src/com/android/dx/io/DexIndexPrinter.java:19: error: package
com.android.dex does not exist

Change-Id: I3217a51c1562194974410c54f936c0713455a93c

croot
cd dalvik
repo start dalvik-is-dead-long-live-dalvik .
repo sync -c .
git rm -r README.txt
git rm -r dexopt
git rm -r tools/deadcode.py
git rm -r tools/dex-preopt
git rm -r tools/dexcheck
git rm -r tools/gdbjithelper
git rm -r unit-tests
git rm -r vm
git checkout HEAD vm/Common.h (needed by libdex)
git checkout HEAD vm/DalvikVersion.h (needed by libdex)
git checkout HEAD vm/Profile.h (needed by dmtracedump)
git add Android.mk (after removing vm, dexopt, and unit-tests references)
git commit -a -m 'Dalvik is dead, long live Dalvik!'

Bug: 14298175
Change-Id: I9dd13053677629d13496d4238af4374452cda415

- The reordering performs during instruction expansion is not correct
for all instructions, in particular for xxx/range instructions that
require sequential registers.

(cherry picked from commit 26a4440a)

Change-Id: Ida3c8a1abfb78a7b119c463b19030572d3fbc40e

This was possible when the first class considered for adding in a dex was too
big to be sure it will fit. Now DX will try to add the class in the current dex
and crash if the class causes one index to overflow.

(cherry picked from commit f15dafe6)

Change-Id: I323f8111c2be6bace908d0539ff324c4bd033c05

Because there were 2 tests 123.

(cherry picked from commit d3fb6889)

Change-Id: Ic6df7590ab32893a40e42ea145d04c64cc265308

- Registers of LocalStart and LocalSnapshot must also be shift when it is required.

Bug: 16344305

(cherry picked from commit 32631e6e)

Change-Id: Ic437ed4102ad0fbeccc069df0f44cb2d774ff27d

Bug: 16344305

(cherry picked from commit 45778955)

Change-Id: Ic607ffda267d8c61ae0a943747b770ac4c8b460d

We no longer need two distinct methods.

Bug: 15563230
Change-Id: Ifd1437b9a8bd0f35a5d2228d2534f69d64d139cc

Change-Id: Ic24de231aea0f4ea49ea51572a839639dc7fb79b

* commit '2c243a10':

* commit '388f055a':
  Align 64-bit registers on even dalvik registers

* commit '388f055a':
  Align 64-bit registers on even dalvik registers