{"id":"streams-files","slug":"files","title":"Reading a file as a stream","text":"Module 20 read files line by line, with a Scanner and a `while` loop. `Files.lines(Paths.get(\"expenses.csv\"))` gives a file's lines as a stream of Strings instead, ready for `filter`, `map` and the rest. `Files` is in `java.nio.file`, next to `Paths`, so it needs an import of its own.\n\n### The lines of a file\n\nThe household budget is now in a file. Each line has a category and an amount in whole dollars, separated by a comma:\n\n```file expenses.csv\ngroceries,54\nrent,850\nbus pass,40\ngroceries,38\nphone,25\n```\n\n```java\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class Main {\n    public static void main(String[] args) {\n        try {\n            Files.lines(Paths.get(\"expenses.csv\"))\n                    .filter(line -> line.startsWith(\"groceries\"))\n                    .forEach(System.out::println);\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n    }\n}\n```\n\n```output\ngroceries,54\ngroceries,38\n```\n\n- The stream's values are the file's lines, in the file's order, without their line breaks.\n- Reading a file can fail, for example when there's no such file. As with the Scanner in module 20, the reading goes in a `try` block, and the `catch` block says what went wrong. For a missing file, the message is the file's name, so the program would print `Error: expenses.csv`.\n\nJava insists on that `try`. Without it, the program doesn't compile:\n\n```java\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class Main {\n    public static void main(String[] args) {\n        Files.lines(Paths.get(\"expenses.csv\")).forEach(System.out::println);\n    }\n}\n```\n\n```javac\nMain.java:6: error: unreported exception IOException; must be caught or declared to be thrown\n        Files.lines(Paths.get(\"expenses.csv\")).forEach(System.out::println);\n                   ^\n1 error\n```\n\nIt's module 20's message about the Scanner, for the same reason.\n\n### Parts of each line\n\nModule 20 split each line with `split(\",\")`. In a stream, that's a `map` from a line to its array of parts, and then the parts can be used:\n\n```file expenses.csv\ngroceries,54\nrent,850\nbus pass,40\ngroceries,38\nphone,25\n```\n\n```java\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class Main {\n    public static void main(String[] args) {\n        try {\n            int total = Files.lines(Paths.get(\"expenses.csv\"))\n                    .map(line -> line.split(\",\"))\n                    .mapToInt(parts -> Integer.valueOf(parts[1]))\n                    .sum();\n            System.out.println(\"Total: \" + total);\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n    }\n}\n```\n\n```output\nTotal: 1007\n```\n\nAfter `map(line -> line.split(\",\"))`, each value in the stream is an array of two Strings, so the next lambda's `parts` is an array: `parts[0]` is the category and `parts[1]` the amount. `total` is created inside the `try` block, so it's printed there too.\n\n### Lines into objects\n\nA `map` whose lambda calls a constructor turns each line into an object, and `collect` gathers them into a list. A method that reads the file this way can return the list, and the rest of the program works on objects:\n\n```file expenses.csv\ngroceries,54\nrent,850\nbus pass,40\n```\n\n```java\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.stream.Collectors;\n\npublic class Main {\n    public static void main(String[] args) {\n        List<Expense> expenses = readExpenses(\"expenses.csv\");\n        System.out.println(expenses);\n        List<Expense> missing = readExpenses(\"august.csv\");\n        System.out.println(missing.size());\n    }\n\n    public static List<Expense> readExpenses(String file) {\n        List<Expense> expenses = new ArrayList<>();\n        try {\n            expenses = Files.lines(Paths.get(file))\n                    .map(line -> line.split(\",\"))\n                    .map(parts -> new Expense(parts[0], Integer.valueOf(parts[1])))\n                    .collect(Collectors.toList());\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n        return expenses;\n    }\n}\n// ==== Expense.java ====\npublic class Expense {\n    private String category;\n    private int amount;\n\n    public Expense(String category, int amount) {\n        this.category = category;\n        this.amount = amount;\n    }\n\n    public String getCategory() {\n        return this.category;\n    }\n\n    public int getAmount() {\n        return this.amount;\n    }\n\n    @Override\n    public String toString() {\n        return this.category + \" \" + this.amount;\n    }\n}\n```\n\n```output\n[groceries 54, rent 850, bus pass 40]\nError: august.csv\n0\n```\n\n- `new Expense(parts[0], Integer.valueOf(parts[1]))` is the lambda's result, so the stream after that `map` is a stream of Expense objects.\n- The method starts with an empty list. If the file can be read, the collected list takes its place; if not, the `catch` block prints the error, and the method returns the empty list, so the rest of the program still has a list to work on.\n\nA stream from `Files.lines` keeps the file open even after its terminal operation: `forEach`, `sum` or `collect` uses the stream up, but doesn't close the file. Module 20's `try` with parentheses closes what it opens at the end of the `try` block, and it works here too, written with the stream's type: `try (Stream<String> lines = Files.lines(Paths.get(file))) { ... }`, with `import java.util.stream.Stream;`. Short programs, like the ones in this module, often leave that out: every file is closed when the program ends.\n","task":"This challenge gives the file `expenses.csv`, shown below, in the same form: the category and the amount, separated by a comma. Print the total of the grocery expenses, the lines whose category is exactly `groceries`. Use `Files.lines` and a stream, in a `try` block. If the file can't be read, print only `Error: ` and the exception's message. For the file shown, the program prints:\n\n```\nGroceries: 92\n```\n","kind":"code","seed":"import java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class Main {\n    public static void main(String[] args) {\n\n    }\n}\n","solution":"import java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class Main {\n    public static void main(String[] args) {\n        try {\n            int groceries = Files.lines(Paths.get(\"expenses.csv\"))\n                    .map(line -> line.split(\",\"))\n                    .filter(parts -> parts[0].equals(\"groceries\"))\n                    .mapToInt(parts -> Integer.valueOf(parts[1]))\n                    .sum();\n            System.out.println(\"Groceries: \" + groceries);\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n    }\n}\n","hints":["Split each line first, `map(line -> line.split(\",\"))`, and then keep the arrays whose first part equals `groceries`, with `equals`, since a category such as `groceries extra` also starts with `groceries`.","Then `mapToInt(parts -> Integer.valueOf(parts[1]))` and `sum()`. Print inside the `try` block: then only the error is printed when the file can't be read."],"tests":[{"name":"Test 1","stdin":"","files":{"expenses.csv":"groceries,54\nrent,850\nbus pass,40\ngroceries,38\nphone,25\n"},"expect":"Groceries: 92","hidden":false},{"name":"Test 2","stdin":"","files":{"expenses.csv":"groceries extra,7\ngroceries,3\ncinema,24\n"},"expect":"Groceries: 3","hidden":true},{"name":"Test 3","stdin":"","files":{"expenses.csv":"rent,850\n"},"expect":"Groceries: 0","hidden":true},{"name":"Test 4","stdin":"","files":{"budget.csv":"groceries,10\n"},"expect":"Error: expenses.csv","hidden":true}],"require":[{"pattern":"\\bFiles\\s*\\.\\s*lines\\s*\\(","message":"Read the file with `Files.lines(Paths.get(\"expenses.csv\"))`."},{"pattern":"\\.sum\\s*\\(\\s*\\)","message":"Add up the amounts with `sum()`."}],"forbid":[{"pattern":"\\b(for|while)\\b","message":"Use a stream this time, not a loop."},{"pattern":"\\bScanner\\b","message":"Read the file with `Files.lines` this time, not a Scanner."}],"more":[{"task":"The bank's export starts with a header line, `category,amount`, before the expenses. Fill in the blank, a stream method, so that the program leaves the header out and prints the number of expenses and their total. For the file shown, it prints:\n\n```\n3 expenses, 136 dollars\n```\n","kind":"fill","seed":"import java.nio.file.Files;\nimport java.nio.file.Paths;\nimport java.util.List;\nimport java.util.stream.Collectors;\n\npublic class Main {\n    public static void main(String[] args) {\n        try {\n            List<Integer> amounts = Files.lines(Paths.get(\"export.csv\"))\n                    .[[skip]](1)\n                    .map(line -> line.split(\",\"))\n                    .map(parts -> Integer.valueOf(parts[1]))\n                    .collect(Collectors.toList());\n            int total = amounts.stream()\n                    .mapToInt(amount -> amount)\n                    .sum();\n            System.out.println(amounts.size() + \" expenses, \" + total + \" dollars\");\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n    }\n}\n","solution":"import java.nio.file.Files;\nimport java.nio.file.Paths;\nimport java.util.List;\nimport java.util.stream.Collectors;\n\npublic class Main {\n    public static void main(String[] args) {\n        try {\n            List<Integer> amounts = Files.lines(Paths.get(\"export.csv\"))\n                    .skip(1)\n                    .map(line -> line.split(\",\"))\n                    .map(parts -> Integer.valueOf(parts[1]))\n                    .collect(Collectors.toList());\n            int total = amounts.stream()\n                    .mapToInt(amount -> amount)\n                    .sum();\n            System.out.println(amounts.size() + \" expenses, \" + total + \" dollars\");\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n    }\n}\n","hints":["`Integer.valueOf(\"amount\")` would crash on the header, so it must be dropped before that `map`. Which intermediate operation drops the first values?","`skip(1)` drops the first line and lets the rest through."],"tests":[{"name":"Test 1","stdin":"","files":{"export.csv":"category,amount\ngroceries,61\nphone,25\nbus pass,50\n"},"expect":"3 expenses, 136 dollars","hidden":false},{"name":"Test 2","stdin":"","files":{"export.csv":"category,amount\nrent,850\n"},"expect":"1 expenses, 850 dollars","hidden":true}],"require":[],"forbid":[]},{"task":"`Expense.java` is ready. In `Main.java`, write the method `expensesIn(String file, String category)`, which reads a file of expenses in the form `category,amount` and returns the expenses of the given category as a `List<Expense>`, in the file's order. The category must match exactly: `groceries extra` is a category of its own, not `groceries`. The file may have empty lines: leave them out. If the file can't be read, the method prints `Error: ` and the exception's message, and returns an empty list. Read the file with `Files.lines` and a stream, with no loops. The check runs, with the file `july.csv` shown below:\n\n```java\nList<Expense> groceries = expensesIn(\"july.csv\", \"groceries\");\nSystem.out.println(groceries);\n```\n\nand expects:\n\n```\n[groceries 54, groceries 38]\n```\n","kind":"code","seed":"// ==== Main.java ====\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.stream.Collectors;\n\npublic class Main {\n    public static void main(String[] args) {\n        List<Expense> groceries = expensesIn(\"july.csv\", \"groceries\");\n        System.out.println(groceries);\n    }\n\n    // Write the method expensesIn here.\n}\n\n// ==== Expense.java ====\npublic class Expense {\n    private String category;\n    private int amount;\n\n    public Expense(String category, int amount) {\n        this.category = category;\n        this.amount = amount;\n    }\n\n    public String getCategory() {\n        return this.category;\n    }\n\n    public int getAmount() {\n        return this.amount;\n    }\n\n    @Override\n    public String toString() {\n        return this.category + \" \" + this.amount;\n    }\n}\n","solution":"// ==== Main.java ====\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.stream.Collectors;\n\npublic class Main {\n    public static void main(String[] args) {\n        List<Expense> groceries = expensesIn(\"july.csv\", \"groceries\");\n        System.out.println(groceries);\n    }\n\n    public static List<Expense> expensesIn(String file, String category) {\n        List<Expense> expenses = new ArrayList<>();\n        try {\n            expenses = Files.lines(Paths.get(file))\n                    .map(line -> line.split(\",\"))\n                    .filter(parts -> parts[0].equals(category))\n                    .map(parts -> new Expense(parts[0], Integer.valueOf(parts[1])))\n                    .collect(Collectors.toList());\n        } catch (Exception e) {\n            System.out.println(\"Error: \" + e.getMessage());\n        }\n        return expenses;\n    }\n}\n\n// ==== Expense.java ====\npublic class Expense {\n    private String category;\n    private int amount;\n\n    public Expense(String category, int amount) {\n        this.category = category;\n        this.amount = amount;\n    }\n\n    public String getCategory() {\n        return this.category;\n    }\n\n    public int getAmount() {\n        return this.amount;\n    }\n\n    @Override\n    public String toString() {\n        return this.category + \" \" + this.amount;\n    }\n}\n","hints":["The header is `public static List<Expense> expensesIn(String file, String category)`. Start with an empty list, replace it with the collected one inside the `try` block, and return it after the `catch` block. A lambda can use the parameter `category`, as step 2 showed.","Split each line first, and keep the arrays whose first part `equals(category)`: an empty line's only part is `\"\"`, so that drops the empty lines too. Then make an Expense of each array and `collect` them."],"tests":[{"name":"Test 1","stdin":"","call":"List<Expense> groceries = expensesIn(\"july.csv\", \"groceries\");\nSystem.out.println(groceries);\n","files":{"july.csv":"groceries,54\nrent,850\ngroceries extra,7\ngroceries,38\nphone,25\n"},"expect":"[groceries 54, groceries 38]","hidden":false},{"name":"Test 2","stdin":"","call":"List<Expense> bus = expensesIn(\"june.csv\", \"bus pass\");\nSystem.out.println(bus.size() + \" \" + bus);\nSystem.out.println(bus.get(1).getCategory() + \" \" + bus.get(1).getAmount());\nSystem.out.println(expensesIn(\"june.csv\", \"groceries\"));\n","files":{"june.csv":"bus pass,40\n\ngroceries,12\nbus pass,35\n\n"},"expect":"2 [bus pass 40, bus pass 35]\nbus pass 35\n[groceries 12]","hidden":true},{"name":"Test 3","stdin":"","call":"System.out.println(expensesIn(\"july.csv\", \"groceries\").size());\nSystem.out.println(expensesIn(\"july.csv\", \"groceries extra\"));\nList<Expense> missing = expensesIn(\"may.csv\", \"rent\");\nSystem.out.println(missing.size());\nSystem.out.println(expensesIn(\"july.csv\", \"rent\"));\n","files":{"july.csv":"rent,850\ngroceries extra,7\n"},"expect":"0\n[groceries extra 7]\nError: may.csv\n0\n[rent 850]","hidden":true}],"require":[{"pattern":"\\bpublic\\s+static\\s+List<Expense>\\s+expensesIn\\s*\\(\\s*String\\s+\\w+\\s*,\\s*String\\s+\\w+\\s*\\)","message":"Write the method as `public static List<Expense> expensesIn(String file, String category)`."},{"pattern":"\\bFiles\\s*\\.\\s*lines\\s*\\(","message":"Read the file with `Files.lines`."}],"forbid":[{"pattern":"\\b(for|while)\\b","message":"Use a stream this time, not a loop."},{"pattern":"\\bScanner\\b","message":"Read the file with `Files.lines` this time, not a Scanner."}]}]}