APIメインのシステムでE2E（エンドツーエンド）テストをするにあたり、なにかいいツールはないかなと探していたら見つけました。その名も、「Karate」。押忍！
Karateとは
Karateは、APIテストの自動化、モック、パフォーマンステスト、さらにはUI自動化を単一の統合フレームワークに組み合わせる唯一のオープンソースツールです。 Cucumberによって一般化されたBDD構文は言語に依存せず、プログラマー以外でも簡単に使用できます。 強力なJSONおよびXMLアサーションが組み込まれており、テストを並行して実行して速度を上げることができます。
SeleniumやCypress、NightwatchなどのUIテストツールに対して、KarateはAPIテストに特化しています。APIだけでなく、UIテストにも対応しているようですね。
Karateをさわってみる
KarateのREADME.mdにしたがって動かしてみます。環境は以下のとおりです。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| - Java 11.0.2 | |
| - Maven 3.6.3 |
by まずは、Karate入りのMavenプロジェクトを作成します。Gradleプロジェクトの場合、こちらのbuild.gradleが参考になります。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ mvn archetype:generate \ | |
| -DarchetypeGroupId=com.intuit.karate \ | |
| -DarchetypeArtifactId=karate-archetype \ | |
| -DarchetypeVersion=0.9.5 \ | |
| -DgroupId=com.mycompany \ | |
| -DartifactId=myproject |
作成されたプロジェクトは以下のようなディレクトリ構成になっています。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| myproject/ | |
| ├── pom.xml | |
| └── src | |
| └── test | |
| └── java | |
| ├── examples | |
| │ ├── ExamplesTest.java # 全シナリオのテスト実行クラス | |
| │ └── users | |
| │ ├── UsersRunner.java # シナリオごとのテスト実行クラス | |
| │ └── users.feature # テストシナリオ | |
| ├── karate-config.js # Karateの設定ファイル | |
| └── logback-test.xml # ロガーの設定 |
ポイントは、src/test/java配下にリソースファイルを配備することです。一般的にMavenやGradleプロジェクトでは、テストに利用するリソースファイル（上でいうと、.jsや.xml）はsrc/test/resources配下に置くのですが、Karateではsrc/test/java配下に配備することを推奨しています。これにより、Karate以外のテストで利用するファイルとごちゃまぜにならないように管理できます。
pom.xmlにsrc/test/java配下にリソースファイルを配備する設定が書かれています。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <build> | |
| <testResources> | |
| <testResource> | |
| <directory>src/test/java</directory> | |
| <excludes> | |
| <exclude>**/*.java</exclude> | |
| </excludes> | |
| </testResource> | |
| </testResources> | |
| <plugins> | |
| ... | |
| </plugins> | |
| </build> |
シナリオの書き方を見てみます。Karateでは、.featureにシナリオを書きます。サンプルとしてすでにusers.featureにシナリオが書かれています。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Feature: sample karate test script | |
| for help, see: https://github.com/intuit/karate/wiki/IDE-Support | |
| Background: | |
| * url 'https://jsonplaceholder.typicode.com' | |
| Scenario: get all users and then get the first user by id | |
| Given path 'users' | |
| When method get | |
| Then status 200 | |
| * def first = response[0] | |
| Given path 'users', first.id | |
| When method get | |
| Then status 200 | |
| # 以下、省略 |
Karateを使ったことがなくても、ぱっと見てなんとなくわかる記述なのがいいですね。具体的には以下のテストシナリオが書かれています。
https://jsonplaceholder.typicode.com/usersにGETでアクセスしたときに、ステータスコードが200であること- レスポンスボディをもとに（
id=1）、https://jsonplaceholder.typicode.com/users/1にGETでアクセスしたときに、ステータスが200であること
APIを呼び出す際、前回のAPI呼び出しの結果を利用できるのがいいですね。記述も簡潔でわかりやすいです。
最後に、テストを動かしてみます。ログも全部載せておきます。
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ mvn test | |
| [INFO] Scanning for projects... | |
| [INFO] | |
| [INFO] ----------------------< com.mycompany:myproject >----------------------- | |
| [INFO] Building myproject 1.0-SNAPSHOT | |
| [INFO] --------------------------------[ jar ]--------------------------------- | |
| [INFO] | |
| [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ myproject --- | |
| [INFO] Using 'UTF-8' encoding to copy filtered resources. | |
| [INFO] skip non existing resourceDirectory /Users/donchan922/Downloads/myproject/src/main/resources | |
| [INFO] | |
| [INFO] --- maven-compiler-plugin:3.6.0:compile (default-compile) @ myproject --- | |
| [INFO] No sources to compile | |
| [INFO] | |
| [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ myproject --- | |
| [INFO] Using 'UTF-8' encoding to copy filtered resources. | |
| [INFO] Copying 3 resources | |
| [INFO] | |
| [INFO] --- maven-compiler-plugin:3.6.0:testCompile (default-testCompile) @ myproject --- | |
| [INFO] Nothing to compile - all classes are up to date | |
| [INFO] | |
| [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ myproject --- | |
| [INFO] | |
| [INFO] ------------------------------------------------------- | |
| [INFO] T E S T S | |
| [INFO] ------------------------------------------------------- | |
| [INFO] Running examples.ExamplesTest | |
| 08:03:32.049 [main] INFO com.intuit.karate - karate.env system property was: | |
| 08:03:32.258 [main] DEBUG com.intuit.karate - request: | |
| 1 > GET https://jsonplaceholder.typicode.com/users | |
| 1 > Accept-Encoding: gzip,deflate | |
| 1 > Connection: Keep-Alive | |
| 1 > Host: jsonplaceholder.typicode.com | |
| 1 > User-Agent: Apache-HttpClient/4.5.11 (Java/13.0.2) | |
| 08:03:32.464 [main] DEBUG com.intuit.karate - response time in milliseconds: 201.42 | |
| 1 < 200 | |
| 1 < Access-Control-Allow-Credentials: true | |
| 1 < Age: 1516 | |
| 1 < CF-Cache-Status: HIT | |
| 1 < CF-RAY: 597a2bcbacc30a20-KIX | |
| 1 < Cache-Control: max-age=14400 | |
| 1 < Connection: keep-alive | |
| 1 < Content-Type: application/json; charset=utf-8 | |
| 1 < Date: Fri, 22 May 2020 23:03:32 GMT | |
| 1 < Etag: W/"160d-1eMSsxeJRfnVLRBmYJSbCiJZ1qQ" | |
| 1 < Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" | |
| 1 < Expires: -1 | |
| 1 < Pragma: no-cache | |
| 1 < Server: cloudflare | |
| 1 < Set-Cookie: __cfduid=de3aae87e65642318311da4d8454044581590188612; expires=Sun, 21-Jun-20 23:03:32 GMT; path=/; domain=.typicode.com; HttpOnly; SameSite=Lax | |
| 1 < Transfer-Encoding: chunked | |
| 1 < Vary: Origin, Accept-Encoding | |
| 1 < Via: 1.1 vegur | |
| 1 < X-Content-Type-Options: nosniff | |
| 1 < X-Powered-By: Express | |
| 1 < cf-request-id: 02e039b34900000a20e9a9f200000001 | |
| [ | |
| { | |
| "id": 1, | |
| "name": "Leanne Graham", | |
| "username": "Bret", | |
| "email": "Sincere@april.biz", | |
| "address": { | |
| "street": "Kulas Light", | |
| "suite": "Apt. 556", | |
| "city": "Gwenborough", | |
| "zipcode": "92998-3874", | |
| "geo": { | |
| "lat": "-37.3159", | |
| "lng": "81.1496" | |
| } | |
| }, | |
| "phone": "1-770-736-8031 x56442", | |
| "website": "hildegard.org", | |
| "company": { | |
| "name": "Romaguera-Crona", | |
| "catchPhrase": "Multi-layered client-server neural-net", | |
| "bs": "harness real-time e-markets" | |
| } | |
| }, | |
| { | |
| "id": 2, | |
| "name": "Ervin Howell", | |
| "username": "Antonette", | |
| "email": "Shanna@melissa.tv", | |
| "address": { | |
| "street": "Victor Plains", | |
| "suite": "Suite 879", | |
| "city": "Wisokyburgh", | |
| "zipcode": "90566-7771", | |
| "geo": { | |
| "lat": "-43.9509", | |
| "lng": "-34.4618" | |
| } | |
| }, | |
| "phone": "010-692-6593 x09125", | |
| "website": "anastasia.net", | |
| "company": { | |
| "name": "Deckow-Crist", | |
| "catchPhrase": "Proactive didactic contingency", | |
| "bs": "synergize scalable supply-chains" | |
| } | |
| }, | |
| { | |
| "id": 3, | |
| "name": "Clementine Bauch", | |
| "username": "Samantha", | |
| "email": "Nathan@yesenia.net", | |
| "address": { | |
| "street": "Douglas Extension", | |
| "suite": "Suite 847", | |
| "city": "McKenziehaven", | |
| "zipcode": "59590-4157", | |
| "geo": { | |
| "lat": "-68.6102", | |
| "lng": "-47.0653" | |
| } | |
| }, | |
| "phone": "1-463-123-4447", | |
| "website": "ramiro.info", | |
| "company": { | |
| "name": "Romaguera-Jacobson", | |
| "catchPhrase": "Face to face bifurcated interface", | |
| "bs": "e-enable strategic applications" | |
| } | |
| }, | |
| { | |
| "id": 4, | |
| "name": "Patricia Lebsack", | |
| "username": "Karianne", | |
| "email": "Julianne.OConner@kory.org", | |
| "address": { | |
| "street": "Hoeger Mall", | |
| "suite": "Apt. 692", | |
| "city": "South Elvis", | |
| "zipcode": "53919-4257", | |
| "geo": { | |
| "lat": "29.4572", | |
| "lng": "-164.2990" | |
| } | |
| }, | |
| "phone": "493-170-9623 x156", | |
| "website": "kale.biz", | |
| "company": { | |
| "name": "Robel-Corkery", | |
| "catchPhrase": "Multi-tiered zero tolerance productivity", | |
| "bs": "transition cutting-edge web services" | |
| } | |
| }, | |
| { | |
| "id": 5, | |
| "name": "Chelsey Dietrich", | |
| "username": "Kamren", | |
| "email": "Lucio_Hettinger@annie.ca", | |
| "address": { | |
| "street": "Skiles Walks", | |
| "suite": "Suite 351", | |
| "city": "Roscoeview", | |
| "zipcode": "33263", | |
| "geo": { | |
| "lat": "-31.8129", | |
| "lng": "62.5342" | |
| } | |
| }, | |
| "phone": "(254)954-1289", | |
| "website": "demarco.info", | |
| "company": { | |
| "name": "Keebler LLC", | |
| "catchPhrase": "User-centric fault-tolerant solution", | |
| "bs": "revolutionize end-to-end systems" | |
| } | |
| }, | |
| { | |
| "id": 6, | |
| "name": "Mrs. Dennis Schulist", | |
| "username": "Leopoldo_Corkery", | |
| "email": "Karley_Dach@jasper.info", | |
| "address": { | |
| "street": "Norberto Crossing", | |
| "suite": "Apt. 950", | |
| "city": "South Christy", | |
| "zipcode": "23505-1337", | |
| "geo": { | |
| "lat": "-71.4197", | |
| "lng": "71.7478" | |
| } | |
| }, | |
| "phone": "1-477-935-8478 x6430", | |
| "website": "ola.org", | |
| "company": { | |
| "name": "Considine-Lockman", | |
| "catchPhrase": "Synchronised bottom-line interface", | |
| "bs": "e-enable innovative applications" | |
| } | |
| }, | |
| { | |
| "id": 7, | |
| "name": "Kurtis Weissnat", | |
| "username": "Elwyn.Skiles", | |
| "email": "Telly.Hoeger@billy.biz", | |
| "address": { | |
| "street": "Rex Trail", | |
| "suite": "Suite 280", | |
| "city": "Howemouth", | |
| "zipcode": "58804-1099", | |
| "geo": { | |
| "lat": "24.8918", | |
| "lng": "21.8984" | |
| } | |
| }, | |
| "phone": "210.067.6132", | |
| "website": "elvis.io", | |
| "company": { | |
| "name": "Johns Group", | |
| "catchPhrase": "Configurable multimedia task-force", | |
| "bs": "generate enterprise e-tailers" | |
| } | |
| }, | |
| { | |
| "id": 8, | |
| "name": "Nicholas Runolfsdottir V", | |
| "username": "Maxime_Nienow", | |
| "email": "Sherwood@rosamond.me", | |
| "address": { | |
| "street": "Ellsworth Summit", | |
| "suite": "Suite 729", | |
| "city": "Aliyaview", | |
| "zipcode": "45169", | |
| "geo": { | |
| "lat": "-14.3990", | |
| "lng": "-120.7677" | |
| } | |
| }, | |
| "phone": "586.493.6943 x140", | |
| "website": "jacynthe.com", | |
| "company": { | |
| "name": "Abernathy Group", | |
| "catchPhrase": "Implemented secondary concept", | |
| "bs": "e-enable extensible e-tailers" | |
| } | |
| }, | |
| { | |
| "id": 9, | |
| "name": "Glenna Reichert", | |
| "username": "Delphine", | |
| "email": "Chaim_McDermott@dana.io", | |
| "address": { | |
| "street": "Dayna Park", | |
| "suite": "Suite 449", | |
| "city": "Bartholomebury", | |
| "zipcode": "76495-3109", | |
| "geo": { | |
| "lat": "24.6463", | |
| "lng": "-168.8889" | |
| } | |
| }, | |
| "phone": "(775)976-6794 x41206", | |
| "website": "conrad.com", | |
| "company": { | |
| "name": "Yost and Sons", | |
| "catchPhrase": "Switchable contextually-based project", | |
| "bs": "aggregate real-time technologies" | |
| } | |
| }, | |
| { | |
| "id": 10, | |
| "name": "Clementina DuBuque", | |
| "username": "Moriah.Stanton", | |
| "email": "Rey.Padberg@karina.biz", | |
| "address": { | |
| "street": "Kattie Turnpike", | |
| "suite": "Suite 198", | |
| "city": "Lebsackbury", | |
| "zipcode": "31428-2261", | |
| "geo": { | |
| "lat": "-38.2386", | |
| "lng": "57.2232" | |
| } | |
| }, | |
| "phone": "024-648-3804", | |
| "website": "ambrose.net", | |
| "company": { | |
| "name": "Hoeger LLC", | |
| "catchPhrase": "Centralized empowering task-force", | |
| "bs": "target end-to-end models" | |
| } | |
| } | |
| ] | |
| 08:03:32.507 [main] DEBUG com.intuit.karate - request: | |
| 2 > GET https://jsonplaceholder.typicode.com/users/1 | |
| 2 > Accept-Encoding: gzip,deflate | |
| 2 > Connection: Keep-Alive | |
| 2 > Cookie: __cfduid=de3aae87e65642318311da4d8454044581590188612 | |
| 2 > Host: jsonplaceholder.typicode.com | |
| 2 > User-Agent: Apache-HttpClient/4.5.11 (Java/13.0.2) | |
| 08:03:32.564 [main] DEBUG com.intuit.karate - response time in milliseconds: 51.84 | |
| 2 < 200 | |
| 2 < Access-Control-Allow-Credentials: true | |
| 2 < Age: 33 | |
| 2 < CF-Cache-Status: HIT | |
| 2 < CF-RAY: 597a2bcc6c470a54-KIX | |
| 2 < Cache-Control: max-age=14400 | |
| 2 < Connection: keep-alive | |
| 2 < Content-Type: application/json; charset=utf-8 | |
| 2 < Date: Fri, 22 May 2020 23:03:32 GMT | |
| 2 < Etag: W/"1fd-+2Y3G3w049iSZtw5t1mzSnunngE" | |
| 2 < Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" | |
| 2 < Expires: -1 | |
| 2 < Pragma: no-cache | |
| 2 < Server: cloudflare | |
| 2 < Transfer-Encoding: chunked | |
| 2 < Vary: Origin, Accept-Encoding | |
| 2 < Via: 1.1 vegur | |
| 2 < X-Content-Type-Options: nosniff | |
| 2 < X-Powered-By: Express | |
| 2 < cf-request-id: 02e039b3bd00000a546d04f200000001 | |
| { | |
| "id": 1, | |
| "name": "Leanne Graham", | |
| "username": "Bret", | |
| "email": "Sincere@april.biz", | |
| "address": { | |
| "street": "Kulas Light", | |
| "suite": "Apt. 556", | |
| "city": "Gwenborough", | |
| "zipcode": "92998-3874", | |
| "geo": { | |
| "lat": "-37.3159", | |
| "lng": "81.1496" | |
| } | |
| }, | |
| "phone": "1-770-736-8031 x56442", | |
| "website": "hildegard.org", | |
| "company": { | |
| "name": "Romaguera-Crona", | |
| "catchPhrase": "Multi-layered client-server neural-net", | |
| "bs": "harness real-time e-markets" | |
| } | |
| } | |
| --------------------------------------------------------- | |
| feature: classpath:examples/users/users.feature | |
| scenarios: 1 | passed: 1 | failed: 0 | time: 0.4796 | |
| --------------------------------------------------------- | |
| HTML report: (paste into browser to view) | Karate version: 0.9.5 | |
| file:/Users/donchan922/Downloads/myproject/target/surefire-reports/examples.users.users.html | |
| --------------------------------------------------------- | |
| [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.234 s - in examples.ExamplesTest | |
| [INFO] | |
| [INFO] Results: | |
| [INFO] | |
| [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 | |
| [INFO] | |
| [INFO] ------------------------------------------------------------------------ | |
| [INFO] BUILD SUCCESS | |
| [INFO] ------------------------------------------------------------------------ | |
| [INFO] Total time: 2.827 s | |
| [INFO] Finished at: 2020-05-23T08:03:33+09:00 | |
| [INFO] ------------------------------------------------------------------------ |
レポートも出力してくれます。見やすいですね。
まとめ
APIのテスト自動化ツール「Karate」を使ってみました。環境構築が簡単で、学習コストも低そうです。この機会にKarateに入門します。押忍!