[rhq-project/rhq] 67ec07: [BZ 185375] fix transitive dependency conflicts
by John Sanda
Branch: refs/heads/master
Home: https://github.com/rhq-project/rhq
Commit: 67ec07dc5b9a86cd09aea803d976e490a073041d
https://github.com/rhq-project/rhq/commit/67ec07dc5b9a86cd09aea803d976e49...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/enterprise/server/installer/pom.xml
Log Message:
-----------
[BZ 185375] fix transitive dependency conflicts
9 years, 2 months
- 1
- 0
[rhq-project/rhq] 2c504a: [BZ 185375] speed up data migration step
by John Sanda
Branch: refs/heads/spinder/BZ185375
Home: https://github.com/rhq-project/rhq
Commit: 2c504afc89a03f513d8e0b6ed44963a6bc6cbf6d
https://github.com/rhq-project/rhq/commit/2c504afc89a03f513d8e0b6ed44963a...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] speed up data migration step
Writes are now done in batches. Reads are done in parallel and concurrently in
batches.
Commit: b9cbea1d29c3463caa8db9dfdef8d559355f5b30
https://github.com/rhq-project/rhq/commit/b9cbea1d29c3463caa8db9dfdef8d55...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/DateUtils.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/Replace412Index.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceRHQ411Index.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/ReplaceIndexTest.java
Log Message:
-----------
[BZ 185375] refactor date functions into util class
Commit: 2d3bdab8506510f21982db25dca71972996036b2
https://github.com/rhq-project/rhq/commit/2d3bdab8506510f21982db25dca7197...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/AbortedException.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceIndex.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/TaskTracker.java
Log Message:
-----------
[BZ 185375] lots of clean up, refactoring, and perf improvements
The TaskTracker class has been copied into the source tree from the
service-metrics maven module. The reason for copying the class is beause the
changes for this BZ are going into a patch release, and I want to minimize the
number of modules that have to be touched for this.
We no longer query the rdbms to load schedule ids. Instead we use the legacy
thrift APIs via the hector library to scan for schedule ids. This way, we query
for only those schedule ids that have data.
Reads are now throttled using a RateLimiter. This is better than using a
semaphore for a couple reasons. First, we do not have to worry about releasing
permits, which had to be done in a couple different places in the code.
Secondly, it is more expressive about the throughput. With a semaphore, it was
hard to determine the read throughput, whereas with RateLimiter it is directly
specified as querys per second.
The migration code previously tried to abort processing if a write failed. This
made the code overly complicated. Since we are now using batches, we only do a
handful of writes per schedule id. We now just have the failure detection and
handling code in the migration finished callback. This simplifies things a lot.
Commit: 8b3a089caf4bc3e6fd1819b46df0de52cb74d05c
https://github.com/rhq-project/rhq/commit/8b3a089caf4bc3e6fd1819b46df0de5...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] add failure detection
There is a now a failure threshold that if exceeded will cause the migration to
be aborted. The rationale is that the read/write rates will likely have to be
adjusted based on each environment. If we are reading too fast for example,
and start generating lots of failiures, it makes sense to go ahead and
terminate the migration and restart with a lower read rate.
Commit: fc4690a2f708f3e9c280ec03a339b39f826e4e1d
https://github.com/rhq-project/rhq/commit/fc4690a2f708f3e9c280ec03a339b39...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
Log Message:
-----------
[BZ 185375] remove key scanning code with hector
I went back and did some testing with the original approach of querying against
all schedule ids that are returned from the RDBMS. I was able to achieve the
same performance (if not better) as I did with hector. I do not see any reason
therefore to pull in additional dependencies.
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: 32de6ed6c65622c0a4edaf506f1fd4c295217c0f
https://github.com/rhq-project/rhq/commit/32de6ed6c65622c0a4edaf506f1fd4c...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] load schedule ids using astyanax
We do not have a good way in C* 1.2 to scan for schedule ids. Instead I am
using netflix's astyanax library which uses the legacy thrift api. It is
using the get_range_slices operation. This should hopefully be faster than
querying every single schedule id returned from the rdbms.
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: 12b8934dedf33d8e324f4d0f8e0aaf9e1962f33b
https://github.com/rhq-project/rhq/commit/12b8934dedf33d8e324f4d0f8e0aaf9...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
Log Message:
-----------
[BZ 185375] initial support for dynamic throttling
Commit: 05d6dd82eabe6f847307e2970978ff50d48b305b
https://github.com/rhq-project/rhq/commit/05d6dd82eabe6f847307e2970978ff5...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
Log Message:
-----------
[BZ 185375] retry data migrations on failures
Commit: 014630bc52d8bd41070758d5e904787c28ce7d80
https://github.com/rhq-project/rhq/commit/014630bc52d8bd41070758d5e904787...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceRHQ411Index.java
Log Message:
-----------
[BZ 185375] use async writes and dynamic throttling for index update
Commit: b279f9431680845a3701ae50a54319fe9acf7d60
https://github.com/rhq-project/rhq/commit/b279f9431680845a3701ae50a54319f...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/MigrateAggregateMetricsTest.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/SchemaUpgradeTest.java
Log Message:
-----------
[BZ 185375] fix regression in MigrateData
Add back check to execute statements when there is not a full batch. Also
updating RateMonitor to include minimum rates.
Commit: d456e79c59ed309ea67171d10caa01a9e3bb2404
https://github.com/rhq-project/rhq/commit/d456e79c59ed309ea67171d10caa01a...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/enterprise/server/appserver/src/main/scripts/rhq-container.build.xml
M modules/enterprise/server/installer/src/main/resources/module/main/module.xml
M pom.xml
Log Message:
-----------
[BZ 185375] adding jboss modules for astyanax
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: 3d766f551a0558d0d4aabb029779a10666ea8f60
https://github.com/rhq-project/rhq/commit/3d766f551a0558d0d4aabb029779a10...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/enterprise/server/appserver/src/main/scripts/rhq-container.build.xml
Log Message:
-----------
[BZ 185375] fix astyanax module dependencies
Compare: https://github.com/rhq-project/rhq/compare/2c504afc89a0^...3d766f551a05
9 years, 2 months
- 1
- 0
[rhq-project/rhq] 870e57: fix eclipse .classpath
by John Mazzitelli
Branch: refs/heads/master
Home: https://github.com/rhq-project/rhq
Commit: 870e5759a8e3d84ccf3592f126a3c3ea595d0004
https://github.com/rhq-project/rhq/commit/870e5759a8e3d84ccf3592f126a3c3e...
Author: John Mazzitelli <mazz(a)redhat.com>
Date: 2015-03-02 (Mon, 02 Mar 2015)
Changed paths:
M .classpath
Log Message:
-----------
fix eclipse .classpath
9 years, 2 months
- 1
- 0
[rhq-project/rhq] 785ca1: [BZ 185375] speed up data migration step
by John Sanda
Branch: refs/heads/master
Home: https://github.com/rhq-project/rhq
Commit: 785ca18f2b2963555ed8e39e17676befcf8c2406
https://github.com/rhq-project/rhq/commit/785ca18f2b2963555ed8e39e17676be...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] speed up data migration step
Writes are now done in batches. Reads are done in parallel and concurrently in
batches.
Commit: 59c745701735920756b7579d9a11a9773bc9ac6f
https://github.com/rhq-project/rhq/commit/59c745701735920756b7579d9a11a97...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/DateUtils.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/Replace412Index.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceRHQ411Index.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/ReplaceIndexTest.java
Log Message:
-----------
[BZ 185375] refactor date functions into util class
Commit: 4d07cd682e5084f81e4171e1265b9f0c22488c38
https://github.com/rhq-project/rhq/commit/4d07cd682e5084f81e4171e1265b9f0...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/AbortedException.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceIndex.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/TaskTracker.java
Log Message:
-----------
[BZ 185375] lots of clean up, refactoring, and perf improvements
The TaskTracker class has been copied into the source tree from the
service-metrics maven module. The reason for copying the class is beause the
changes for this BZ are going into a patch release, and I want to minimize the
number of modules that have to be touched for this.
We no longer query the rdbms to load schedule ids. Instead we use the legacy
thrift APIs via the hector library to scan for schedule ids. This way, we query
for only those schedule ids that have data.
Reads are now throttled using a RateLimiter. This is better than using a
semaphore for a couple reasons. First, we do not have to worry about releasing
permits, which had to be done in a couple different places in the code.
Secondly, it is more expressive about the throughput. With a semaphore, it was
hard to determine the read throughput, whereas with RateLimiter it is directly
specified as querys per second.
The migration code previously tried to abort processing if a write failed. This
made the code overly complicated. Since we are now using batches, we only do a
handful of writes per schedule id. We now just have the failure detection and
handling code in the migration finished callback. This simplifies things a lot.
Commit: d4d8743e6364ef9078f40700391d9bcacd2d97ce
https://github.com/rhq-project/rhq/commit/d4d8743e6364ef9078f40700391d9bc...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] add failure detection
There is a now a failure threshold that if exceeded will cause the migration to
be aborted. The rationale is that the read/write rates will likely have to be
adjusted based on each environment. If we are reading too fast for example,
and start generating lots of failiures, it makes sense to go ahead and
terminate the migration and restart with a lower read rate.
Commit: 966e923f6fff5c939d0df9c1b74bb9019615155e
https://github.com/rhq-project/rhq/commit/966e923f6fff5c939d0df9c1b74bb90...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
Log Message:
-----------
[BZ 185375] remove key scanning code with hector
I went back and did some testing with the original approach of querying against
all schedule ids that are returned from the RDBMS. I was able to achieve the
same performance (if not better) as I did with hector. I do not see any reason
therefore to pull in additional dependencies.
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: 68313f3b163e8e627079730ec42bb7e8020b6980
https://github.com/rhq-project/rhq/commit/68313f3b163e8e627079730ec42bb7e...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
Log Message:
-----------
[BZ 185375] load schedule ids using astyanax
We do not have a good way in C* 1.2 to scan for schedule ids. Instead I am
using netflix's astyanax library which uses the legacy thrift api. It is
using the get_range_slices operation. This should hopefully be faster than
querying every single schedule id returned from the rdbms.
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: c5ae07fd1e2b1f9ced416de148f9ad7d1d77fb6a
https://github.com/rhq-project/rhq/commit/c5ae07fd1e2b1f9ced416de148f9ad7...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
A modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
Log Message:
-----------
[BZ 185375] initial support for dynamic throttling
Commit: f6b2fc9dd3d8cec578c130ebe569cf537337add5
https://github.com/rhq-project/rhq/commit/f6b2fc9dd3d8cec578c130ebe569cf5...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
Log Message:
-----------
[BZ 185375] retry data migrations on failures
Commit: 4d87b270289ce8c402626fc7281c2415f71ee11b
https://github.com/rhq-project/rhq/commit/4d87b270289ce8c402626fc7281c241...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/ReplaceRHQ411Index.java
Log Message:
-----------
[BZ 185375] use async writes and dynamic throttling for index update
Commit: cb4b4f38029ae8b83147beebfe30ca1bbadaf266
https://github.com/rhq-project/rhq/commit/cb4b4f38029ae8b83147beebfe30ca1...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateAggregateMetrics.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/MigrateData.java
M modules/common/cassandra-schema/src/main/java/org/rhq/cassandra/schema/RateMonitor.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/MigrateAggregateMetricsTest.java
M modules/common/cassandra-schema/src/test/java/org/rhq/cassandra/schema/SchemaUpgradeTest.java
Log Message:
-----------
[BZ 185375] fix regression in MigrateData
Add back check to execute statements when there is not a full batch. Also
updating RateMonitor to include minimum rates.
Commit: 90d9837d5bda4e0a3891f15176f49f62320a3f2d
https://github.com/rhq-project/rhq/commit/90d9837d5bda4e0a3891f15176f49f6...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/enterprise/server/appserver/src/main/scripts/rhq-container.build.xml
M modules/enterprise/server/installer/src/main/resources/module/main/module.xml
M pom.xml
Log Message:
-----------
[BZ 185375] adding jboss modules for astyanax
Conflicts:
modules/common/cassandra-schema/pom.xml
Commit: e4989b1501113fd4eff56eed2246b95615a089c0
https://github.com/rhq-project/rhq/commit/e4989b1501113fd4eff56eed2246b95...
Author: John Sanda <jsanda(a)redhat.com>
Date: 2015-03-01 (Sun, 01 Mar 2015)
Changed paths:
M modules/common/cassandra-schema/pom.xml
M modules/enterprise/server/appserver/src/main/scripts/rhq-container.build.xml
Log Message:
-----------
[BZ 185375] fix astyanax module dependencies
Compare: https://github.com/rhq-project/rhq/compare/289920aeb8d8...e4989b150111
9 years, 2 months
- 1
- 0