Aug
10
DynamoDB – Global Secondary Indexes ”; Previous Next Applications requiring various query types with different attributes can use a single or multiple global secondary indexes in performing these detailed queries. For example − A system keeping a track of users, their login status, and their time logged in. The growth of the previous example slows queries on its data. Global secondary indexes accelerate queries by organizing a selection of attributes from a table. They employ primary keys in sorting data, and require no key table attributes, or key schema identical to the table. All the global secondary indexes must include a partition key, with the option of a sort key. The index key schema can differ from the table, and index key attributes can use any top-level string, number, or binary table attributes. In a projection, you can use other table attributes, however, queries do not retrieve from parent tables. Attribute Projections Projections consist of an attribute set copied from table to secondary index. A Projection always occurs with the table partition key and sort key. In queries, projections allow DynamoDB access to any attribute of the projection; they essentially exist as their own table. In a secondary index creation, you must specify attributes for projection. DynamoDB offers three ways to perform this task − KEYS_ONLY − All index items consist of table partition and sort key values, and index key values. This creates the smallest index. INCLUDE − It includes KEYS_ONLY attributes and specified non-key attributes. ALL − It includes all source table attributes, creating the largest possible index. Note the tradeoffs in projecting attributes into a global secondary index, which relate to throughput and storage cost. Consider the following points − If you only need access to a few attributes, with low latency, project only those you need. This reduces storage and write costs. If an application frequently accesses certain non-key attributes, project them because the storage costs pale in comparison to scan consumption. You can project large sets of attributes frequently accessed, however, this carries a high storage cost. Use KEYS_ONLY for infrequent table queries and frequent writes/updates. This controls size, but still offers good performance on queries. Global Secondary Index Queries and Scans You can utilize queries for accessing a single or multiple items in an index. You must specify index and table name, desired attributes, and conditions; with the option to return results in ascending or descending order. You can also utilize scans to get all index data. It requires table and index name. You utilize a filter expression to retrieve specific data. Table and Index Data Synchronization DynamoDB automatically performs synchronization on indexes with their parent table. Each modifying operation on items causes asynchronous updates, however, applications do not write to indexes directly. You need to understand the impact of DynamoDB maintenance on indices. On creation of an index, you specify key attributes and data types, which means on a write, those data types must match key schema data types. On item creation or deletion, indexes update in an eventually consistent manner, however, updates to data propagate in a fraction of a second (unless system failure of some type occurs). You must account for this delay in applications. Throughput Considerations in Global Secondary Indexes − Multiple global secondary indexes impact throughput. Index creation requires capacity unit specifications, which exist separate from the table, resulting in operations consuming index capacity units rather than table units. This can result in throttling if a query or write exceeds provisioned throughput. View throughput settings by using DescribeTable. Read Capacity − Global secondary indexes deliver eventual consistency. In queries, DynamoDB performs provision calculations identical to that used for tables, with a lone difference of using index entry size rather than item size. The limit of a query returns remains 1MB, which includes attribute name size and values across every returned item. Write Capacity When write operations occur, the affected index consumes write units. Write throughput costs are the sum of write capacity units consumed in table writes and units consumed in index updates. A successful write operation requires sufficient capacity, or it results in throttling. Write costs also remain dependent on certain factors, some of which are as follows − New items defining indexed attributes or item updates defining undefined indexed attributes use a single write operation to add the item to the index. Updates changing indexed key attribute value use two writes to delete an item and write a new one. A table write triggering deletion of an indexed attribute uses a single write to erase the old item projection in the index. Items absent in the index prior to and after an update operation use no writes. Updates changing only projected attribute value in the index key schema, and not indexed key attribute value, use one write to update values of projected attributes into the index. All these factors assume an item size of less than or equal to 1KB. Global Secondary Index Storage On an item write, DynamoDB automatically copies the right set of attributes to any indices where the attributes must exist. This impacts your account by charging it for table item storage and attribute storage. The space used results from the sum of these quantities − Byte size of table primary key Byte size of index key attribute Byte size of projected attributes 100 byte-overhead per index item You can estimate storage needs through estimating average item size and multiplying by the quantity of the table items with the global secondary index key attributes. DynamoDB does not write item data for a table item with an undefined attribute defined as an index partition or sort key. Global Secondary Index Crud Create a table with global secondary indexes by using the CreateTable operation paired with the GlobalSecondaryIndexes parameter. You must specify an attribute to serve as the index partition key, or use another for the index sort key. All index key attributes must be string, number, or binary scalars. You must