my opinion is my own

DBpediaのデータ(turtle)をwgetで一括ダウンロード

日本語版のDBpediaの現時点の最新版である http://ja.dbpedia.org/dumps/20160407/ 配下の全データをwgetで再帰的に取得しています。「-np」は親ディレクトリを取得対象しないようにするためのオプションです。

wget -r -np -R "index.html?*" http://ja.dbpedia.org/dumps/20160407/

取得結果

[ec2-user@bastin 20160407]$ pwd
/home/ec2-user/dbpedia/ja.dbpedia.org/dumps/20160407
[ec2-user@bastin 20160407]$ du -sh
4.4G	.
[ec2-user@bastin 20160407]$ ls -l
total 4306268
-rw-rw-r-- 1 ec2-user ec2-user      16381 Feb 25 01:26 index.html
-rw-rw-r-- 1 ec2-user ec2-user   64620507 Apr 27  2016 jawiki-20160407-article-categories.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   36087466 Apr 27  2016 jawiki-20160407-article-categories.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   11199701 Apr 27  2016 jawiki-20160407-article-templates-nested.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    5854138 Apr 27  2016 jawiki-20160407-article-templates-nested.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   58951338 Apr 27  2016 jawiki-20160407-article-templates.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   29388098 Apr 27  2016 jawiki-20160407-article-templates.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    2441006 Apr 27  2016 jawiki-20160407-category-labels.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    1416295 Apr 27  2016 jawiki-20160407-category-labels.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    2047854 Apr 27  2016 jawiki-20160407-disambiguations-unredirected.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    1258154 Apr 27  2016 jawiki-20160407-disambiguations-unredirected.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   35396297 Apr 27  2016 jawiki-20160407-external-links.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   24184031 Apr 27  2016 jawiki-20160407-external-links.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user     586677 Apr 27  2016 jawiki-20160407-geo-coordinates-mappingbased.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user     390558 Apr 27  2016 jawiki-20160407-geo-coordinates-mappingbased.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    1052110 Apr 27  2016 jawiki-20160407-geo-coordinates.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user     710988 Apr 27  2016 jawiki-20160407-geo-coordinates.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    3242564 Apr 27  2016 jawiki-20160407-homepages.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    2018246 Apr 27  2016 jawiki-20160407-homepages.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   25713177 Apr 27  2016 jawiki-20160407-images.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   18660941 Apr 27  2016 jawiki-20160407-images.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user  146867252 Apr 27  2016 jawiki-20160407-infobox-properties-unredirected.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user  106114414 Apr 27  2016 jawiki-20160407-infobox-properties-unredirected.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user     539768 Apr 27  2016 jawiki-20160407-infobox-property-definitions.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user     257668 Apr 27  2016 jawiki-20160407-infobox-property-definitions.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user         91 Apr 27  2016 jawiki-20160407-infobox-test.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user         92 Apr 27  2016 jawiki-20160407-infobox-test.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    6749954 Apr 27  2016 jawiki-20160407-instance-types.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   20605723 Apr 27  2016 jawiki-20160407-instance-types-transitive.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   12598132 Apr 27  2016 jawiki-20160407-instance-types-transitive.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    3302762 Apr 27  2016 jawiki-20160407-instance-types.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user     777393 Apr 27  2016 jawiki-20160407-interlanguage-links.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user     482555 Apr 27  2016 jawiki-20160407-interlanguage-links.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   30962284 Apr 27  2016 jawiki-20160407-labels.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   19374825 Apr 27  2016 jawiki-20160407-labels.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   23766215 Apr 27  2016 jawiki-20160407-mappingbased-literals.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   14463439 Apr 27  2016 jawiki-20160407-mappingbased-literals.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   28382739 Apr 27  2016 jawiki-20160407-mappingbased-objects-uncleaned-unredirected.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   16038031 Apr 27  2016 jawiki-20160407-mappingbased-objects-uncleaned-unredirected.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   26566377 Apr 27  2016 jawiki-20160407-out-degree.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   14217551 Apr 27  2016 jawiki-20160407-out-degree.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   34738422 Apr 27  2016 jawiki-20160407-page-ids.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   19564834 Apr 27  2016 jawiki-20160407-page-ids.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   28049111 Apr 27  2016 jawiki-20160407-page-length.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   15491318 Apr 27  2016 jawiki-20160407-page-length.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user  569374886 Apr 27  2016 jawiki-20160407-page-links-unredirected.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user  367564292 Apr 27  2016 jawiki-20160407-page-links-unredirected.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   33225236 Apr 27  2016 jawiki-20160407-page_props.sql.gz
-rw-rw-r-- 1 ec2-user ec2-user 2285740564 Apr 27  2016 jawiki-20160407-pages-articles.xml.bz2
-rw-rw-r-- 1 ec2-user ec2-user   12735146 Apr 27  2016 jawiki-20160407-redirects.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    8214822 Apr 27  2016 jawiki-20160407-redirects.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   36232990 Apr 27  2016 jawiki-20160407-revision-ids.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   22263076 Apr 27  2016 jawiki-20160407-revision-ids.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   38797873 Apr 27  2016 jawiki-20160407-revision-uris.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   29276720 Apr 27  2016 jawiki-20160407-revision-uris.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user    7261254 Apr 27  2016 jawiki-20160407-skos-categories.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user    4328988 Apr 27  2016 jawiki-20160407-skos-categories.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user     751735 Apr 27  2016 jawiki-20160407-specific-mappingbased-properties.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user     381452 Apr 27  2016 jawiki-20160407-specific-mappingbased-properties.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user      12460 Apr 27  2016 jawiki-20160407-topical-concepts-unredirected.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user       7402 Apr 27  2016 jawiki-20160407-topical-concepts-unredirected.ttl.bz2
-rw-rw-r-- 1 ec2-user ec2-user   60923158 Apr 27  2016 jawiki-20160407-wikipedia-links.tql.bz2
-rw-rw-r-- 1 ec2-user ec2-user   37255115 Apr 27  2016 jawiki-20160407-wikipedia-links.ttl.bz2
drwxrwxr-x 2 ec2-user ec2-user        214 Feb 25 01:28 links
[ec2-user@bastin 20160407]$ 
---

関連しているかもしれない記事


#DBpedia