crt0mega
/
mvt
mirror of https://github.com/mvt-project/mvt.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
944 Commits 22 Branches 67 Tags 6 MiB
Commit Graph

944 Commits

Author SHA1 Message Date
Donncha Ó Cearbhaill 829a9f0cf6
Merge pull request #371 from mvt-project/add-coverage 
Add code test coverage reporting using pytest-cov
 2023-07-22 19:56:04 +02:00
Donncha Ó Cearbhaill 52e0176d5d Add code test coverage reporting 2023-07-22 19:54:01 +02:00
Donncha Ó Cearbhaill 8d8bdf26de Fix black style checks 2023-07-22 19:52:25 +02:00
Donncha Ó Cearbhaill 34fa77ae4d Add documentation for new options 2023-07-22 19:49:59 +02:00
Donncha Ó Cearbhaill ed7d6fb847 Add integration tests for 'mvt-android check-backup' 2023-07-22 19:26:05 +02:00
Donncha Ó Cearbhaill a2386dbdf7 Refactor Android backup password handling and add tests 2023-07-22 19:17:27 +02:00
Donncha Ó Cearbhaill 019cfbb84e
Merge pull request #363 from aticu/main 
Add option to disable interactivity and pass Android backup password on CLI
 2023-07-22 16:44:35 +02:00
Donncha Ó Cearbhaill 3d924e22ec Merge branch 'release/v2.4.0' 2023-07-21 12:17:32 +02:00
Donncha Ó Cearbhaill ca3c1bade4 Bump version to v2.4.0 
Bumping the minor version as we introduce some backwards-incompatible
API changes to module definition in #367.
 2023-07-21 12:14:31 +02:00
Donncha Ó Cearbhaill 85877fd3eb
Merge pull request #369 from mvt-project/move-indicator-checking 
Move detection and alerts from run() to check_indicators()
 2023-07-21 12:12:36 +02:00
Donncha Ó Cearbhaill 8015ff78e8 Fix black error 2023-07-21 12:10:45 +02:00
Donncha Ó Cearbhaill 1a07b9a78f Move syntax checking before unit tests 2023-07-21 11:30:59 +02:00
Donncha Ó Cearbhaill 0b88de9867 Move detection and alerts from run() to check_indicators() 2023-07-21 11:29:12 +02:00
Niclas Schwarzlose 0edc9d7b81 Add option to disable interactivity 2023-07-19 11:29:51 +02:00
Donncha Ó Cearbhaill 76d7534b05 Fix bug recording detections in WebkitResourceLoadStatistics module 2023-07-18 18:02:42 +02:00
Donncha Ó Cearbhaill ae2ab02347
Merge pull request #367 from mvt-project/refactor-module-options 
Add a module_options parameter to pass data from CLI to modules
 2023-07-17 19:07:41 +02:00
Donncha Ó Cearbhaill e2c623c40f Move --fast flag from being a top-level MVT module parameter to an option in a new module_options parameter 2023-07-17 18:52:35 +02:00
Christian Clauss a6e1a3de12
Add GitHub Annotions to ruff output (#364) 
* Add GitHub Annotions to ruff output
* Upgrade GitHub Actions
* No Py3.11
 2023-07-15 14:42:13 +02:00
tek e7270d6a07 Fixes import and adds test for PR 361 2023-07-10 22:55:22 +02:00
Niclas Schwarzlose 1968a0fca2
Improve appops parsing in dumpsys (#361) 
Without this change the package doesn't get properly reset when a new
user starts.

See for example in this excerpt:

```
 1 |    Package com.android.bluetooth:
 2 |      READ_CONTACTS (allow):
 3 |        null=[
 4 |          Access: [pers-s] 2022-04-22 13:24:17.577 (-277d5h22m53s447ms)
 5 |        ]
 6 |      WAKE_LOCK (allow):
 7 |        null=[
 8 |          Access: [pers-s] 2023-01-24 17:45:49.712 (-1m21s312ms) duration=+3ms
 9 |        ]
10 |      GET_USAGE_STATS (default):
11 |        null=[
12 |          Reject: [pers-s]2022-04-22 13:23:53.964 (-277d5h23m17s60ms)
13 |        ]
14 |      BLUETOOTH_CONNECT (allow):
15 |        null=[
16 |          Access: [pers-s] 2022-04-22 13:23:53.988 (-277d5h23m17s36ms)
17 |        ]
18 |  Uid 1027:
19 |    state=pers
20 |    capability=LCMN
21 |    appWidgetVisible=false
22 |      LEGACY_STORAGE: mode=ignore
23 |    Package com.android.nfc:
24 |      WAKE_LOCK (allow):
25 |        null=[
26 |          Access: [pers-s] 2022-04-22 13:23:54.633 (-277d5h23m16s391ms) duration=+1s73ms
27 |        ]
```

Here the package "com.android.bluetooth" is not reset when in line 18,
so when "LEGACY_STORAGE:" in line 22 is encountered, it's added as
another permission to "com.android.bluetooth" with "access" set to
"ode=igno".

This PR fixes that by resetting the package whenever a new Uid is
encountered.

Co-authored-by: Niclas Schwarzlose <niclas.schwarzlose@reporter-ohne-grenzen.de>
 2023-07-10 22:53:58 +02:00
Donncha Ó Cearbhaill 46cc54df74 Add information about public indicators and support avenues to documentation 2023-06-30 19:43:30 +02:00
Donncha Ó Cearbhaill 7046ff80d1 Add SMS read time in the MVT logs 2023-06-30 19:30:50 +02:00
Donncha Ó Cearbhaill e2516f284b
Bump version number 2023-06-29 17:03:26 +02:00
Donncha Ó Cearbhaill 17963f83d6 Fix URL to indicator repo in docs 2023-06-29 16:49:20 +02:00
Donncha Ó Cearbhaill 4f0c9c6077 Update README with information on indicators of compromise and path ways for forensic support 2023-06-29 16:48:56 +02:00
Donncha Ó Cearbhaill 27bd5f03a8
Merge pull request #359 from mvt-project/optimise-domain-checking 
Optimise domain checking performance
 2023-06-29 14:56:50 +02:00
Donncha Ó Cearbhaill 3babbadc1d Add docs for the profiling feature 2023-06-29 14:55:09 +02:00
Donncha Ó Cearbhaill 41db117168 Improve performance when checking URLs and domains 
Some MVT modules such as the WhatsApp module can be very slow as it was taking a naive approach to look for IOCs. The code was checking URLs (potentially more than 100k) against
1000's of IOC domains resulting in a quadratic run-time with hundreds of millions of comparisons as the number of IOCs increases.

This commit add an Aho-Corasick library which allows the efficient search in a string (the URL in this case) for all matches in set of keys (the IOCs). This data structure is perfect for this use case.

A quick measurement shows a 80% performance improvement for a WhatsApp database with 100k entries. The slow path is now the time spent fetching and expanding short URLs found in the database. This
can also be sped up significantly by fetching each URL asynchronously. This would require reworking modules to split the URL expansion from the IOC check so I will implement in a separate PR.
 2023-06-29 14:14:44 +02:00
Donncha Ó Cearbhaill 2b01ed7179 Add optional profiling for MVT modules 2023-06-29 13:31:13 +02:00
Donncha Ó Cearbhaill 78d493b17e
Merge pull request #356 from mvt-project/auto/add-new-ios-releases 
[auto] Update iOS releases and versions
 2023-06-22 11:06:45 +02:00
DonnchaC 473c80009b Add new iOS versions and build numbers 2023-06-22 00:17:52 +00:00
tek a1481683e3 Adapts linter workflow to black 2023-06-14 01:05:14 +02:00
Nex bdd36a9179
Merge pull request #349 from mvt-project/code-cleanup 
Linted code using isort + autoflake + black
 2023-06-08 21:12:34 +02:00
Nex e1677639c4 Linted code using isort + autoflake + black, fixed wrong use of Optional[bool] 2023-06-01 23:40:26 +02:00
tek c2d740ed36 Handle better some empty database issues in iOS backups 2023-05-25 00:24:34 +02:00
tek d0e24c6369 Fixes a bug in the applications module 2023-05-24 12:04:03 +02:00
tek a1994079b1 Sort imports 2023-05-24 12:03:49 +02:00
Donncha Ó Cearbhaill 289b7efdeb Add missing iOS build numbers 2023-05-21 17:11:07 +01:00
Donncha Ó Cearbhaill 166a63e14c
Merge pull request #347 from mvt-project/auto/add-new-ios-releases 
[auto] Update iOS releases and versions
 2023-05-21 17:54:25 +02:00
DonnchaC 1b933fdb12 Add new iOS versions and build numbers 2023-05-21 15:53:45 +00:00
Donncha Ó Cearbhaill 0c0ff7012b Set branch number for auto-generated pull request 2023-05-21 16:52:47 +01:00
Donncha Ó Cearbhaill f9b0d07a81 Don't include information beta's in the version JSON 2023-05-21 16:49:14 +01:00
Donncha Ó Cearbhaill d14bcdd05f Update title used in auto PR for new iOS versions 2023-05-21 16:47:56 +01:00
Donncha Ó Cearbhaill e026bb0a76 Fix path to script in workflow 2023-05-21 16:44:17 +01:00
Donncha Ó Cearbhaill 253b4f031a Allow workflow to be triggered manually 2023-05-21 16:42:54 +01:00
Donncha Ó Cearbhaill ec14297643
Merge pull request #345 from mvt-project/feature/auto-update-version-info 
Add workflow to auto-update iOS builds and version numbers
 2023-05-21 17:38:46 +02:00
Donncha Ó Cearbhaill 3142d86edd Fix path to include version JSON files in built package 2023-05-21 16:37:36 +01:00
Donncha Ó Cearbhaill c18998d771 Add version 16.5 to resolve merge conflict from main 2023-05-21 16:26:12 +01:00
Donncha Ó Cearbhaill 22fd794fb8 Fix python style and setup.cfg syntax 2023-05-21 16:15:49 +01:00
Donncha Ó Cearbhaill 27c5c76dc2 Add script and worker to auto-update build and version info 2023-05-21 16:09:50 +01:00