robots.txt
2 rows • 1 columns • , delimited
Key Data Insights
- Contains 2 rows of structured data.
- Includes columns covering User-agent: * (categorical)...
- Fully analyzed and categorized for open data research.
Column Dictionary & Summary
| Column Name | Type | Missing | Distinct | Description |
|---|---|---|---|---|
User-agent: * |
categorical | 0.0% missing | 2 distinct | categorical data field: User agent: * |
Column Distributions & Statistics
User-agent: * categorical
categorical data field: User agent: *
Distinct Values: 2 • Top Value: # See http://www.robotstxt.org/wc/norobots.html… (50%)