Appearance
CreateIndex
Use this API to create two types of knowledge bases: unstructured knowledge bases for documents, audio, or video, and structured knowledge bases for data queries or image Q&A.
Operation description
Permissions:
RAM user : A RAM user must obtain the required API permissions for Alibaba Cloud Model Studio and join a workspace before calling this operation. You can use the
AliyunBailianDataFullAccesspolicy, which includes the `sfm:CreateIndex` permission required for this operation.Alibaba Cloud account: By default, Alibaba Cloud accounts have the required permissions and can call this operation directly.
How to call : Use the latest version of the Alibaba Cloud Model Studio SDK to call this operation. The SDK handles the complex signature calculation logic to simplify the process.
What to do next : This operation only initializes a knowledge base creation job. After calling this operation, you must call the SubmitIndexJob operation to complete the creation. Otherwise, an empty knowledge base is created. For related code examples, see Knowledge base API guide.
Idempotence: This operation is not idempotent. Repeated calls may create multiple knowledge bases with the same name. To ensure idempotence, query for the knowledge base before you create it.
Rate limiting: Frequent calls to this operation are subject to rate limiting. Do not exceed a frequency of 10 calls per second. If you encounter rate limiting, retry the call later.
Try it now
Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage. Test
RAM authorization
The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:
Action: The actions can be used in the
Actionelement of RAM permission policy statements to grant permissions to perform the operation.API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the
Resourceelement of the policy.For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (***** ) in the
Resourceelement of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.
| Action | Access level | Resource type | Condition key | Dependent action |
|---|---|---|---|---|
| sfm:CreateIndex | create | *All Resource* | None | None |
Request syntax
HELPCODEESCAPE-plaintext
POST /{WorkspaceId}/index/create HTTP/1.1Path Parameters
| Parameter | Type | Required | Description | Example |
|---|---|---|---|---|
| WorkspaceId | string | Yes | The ID of the workspace where you want to create the knowledge base. For more information about how to obtain the workspace ID, see How to use workspaces. | llm-3z7uw7fwz0vexxxx |
Request parameters
| Parameter | Type | Required | Description | Example |
|---|---|---|---|---|
| Name | string | Yes | The name of the knowledge base. The name can be 1 to 20 characters in length and can contain Chinese characters, letters, digits, underscores (_), hyphens (-), periods (.), and colons (😃. | Enterprise doc lib |
| StructureType | string | Yes | The type of the knowledge base. Valid values: - unstructured: A knowledge base for document search, audio, or video. The default scenario for document search is basic document Q&A. ** **Note ** The type of a knowledge base cannot be changed after it is created. Valid values: - unstructured : unstructured | unstructured |
| EmbeddingModelName | string | No | - The vector model used by the knowledge base. A vector model converts the original input prompt and knowledge text into numerical vectors to compare their similarity. The default text-embedding-v2 model (which cannot be changed for now) supports both Chinese and English, along with multiple other languages, and normalizes the vector results. For more information, see Vectorization. Valid values: - text-embedding-v2 If you do not specify this parameter, `text-embedding-v2` is used. | text-embedding-v2 |
| RerankModelName | string | No | The reranking model used by the knowledge base. The reranking model is an external scoring system that calculates a similarity score between the user's question and each text chunk in the knowledge base, sorts them in descending order, and returns the top K text chunks. Valid values: - gte-rerank-hybrid: Official reranking. - gte-rerank: gte-rerank reranking. If you do not specify this parameter, `gte-rerank-hybrid` is used. ** **Note ** Use `gte-rerank` if you only need semantic sorting. Use `gte-rerank-hybrid` if you need both semantic sorting and text-matching features to ensure relevance. Valid values: - gte-rerank-hybrid : Official reranking - gte-rerank : gte-rerank reranking | gte-rerank-hybrid |
| RerankMinScore | number | No | The similarity threshold. Only text chunks with a similarity score greater than this value are recalled. This is used to filter the text chunks returned by the reranking model. The value must be between 0.01 and 1.00. If you do not specify this parameter, the default value 0.01 is used. | 0.20 |
| ChunkSize | integer | No | The chunk size, which is the maximum number of characters for each text chunk. If this length is exceeded, the text is likely to be truncated. The value must be between 1 and 6000. If you do not specify this parameter, the default value 500 is used. ** **Note ** If you set `ChunkSize` to a value less than 100, you must also set `OverlapSize`. You can also leave both parameters unspecified, and the system will use the default values. | 128 |
| OverlapSize | integer | No | The overlap size, which is the number of overlapping characters between the current text chunk and the previous one. The value must be between 0 and 1024. If you do not specify this parameter, the default value 100 is used. ** **Note ** `OverlapSize` must be smaller than `ChunkSize`. Otherwise, chunking errors will occur. | 16 |
| Separator | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | (?<=。) |
| SourceType | string | No | ** **Important ** In the latest SDK version, this parameter is required. Otherwise, calling the SubmitIndexJob operation will result in the error: Required parameter(data_sources) missing or invalid. The source of the imported data. Valid values: - DATA_CENTER_CATEGORY: Category type. Imports all files under the specified categories in Application Data. You can import multiple categories at the same time. - DATA_CENTER_FILE: File type. Imports the specified files from Application Data. You can import multiple files at the same time. ** **Note ** If you set this parameter to DATA_CENTER_CATEGORY, you must specify the `CategoryIds` parameter. If you set this parameter to DATA_CENTER_FILE, you must specify the `DocumentIds` parameter. ** **Note ** To create an empty knowledge base, use an empty category that contains no files. Set this parameter to DATA_CENTER_CATEGORY and specify the ID of the empty category for `CategoryIds`. Valid values: - DATA_CENTER_CATEGORY : Category type - DATA_CENTER_FILE : File type | DATA_CENTER_FILE |
| DocumentIds | array | No | You can import files when you create a knowledge base. Specify a list of files to import by providing their IDs. We recommend importing no more than 10,000 files. If you have more files, you can call the SubmitIndexAddDocumentsJob operation to import them later. | |
| string | No | The file ID. This is the FileId returned by the AddFile operation. You can also obtain the ID by clicking the ID icon next to the file name in the file connector under Data Connectors or on the Files tab of Application Data. | file_9a65732555b54d5ea10796ca5742ba22_xxxxxxxx | |
| CategoryIds | array | No | You can import files when you create a knowledge base. Specify category IDs to import all files under the corresponding categories. We recommend importing no more than 10,000 files. If you have more files, you can call the SubmitIndexAddDocumentsJob operation to import them later. | |
| string | No | The category ID. This is the `CategoryId` returned by the AddCategory operation. You can also obtain it on the Files tab of Application Data by clicking the ID icon next to the category name. | ca_hiu2383nfxxxx | |
| TableIds | array | No | ** **Note ** This parameter is not yet available. Do not specify it. | |
| string | No | |||
| SinkType | string | Yes | The storage class for the knowledge base vectors. For more information, see Knowledge bases. Valid values: - BUILT_IN: Hosts the vector data on the Alibaba Cloud Model Studio platform. - ADB: AnalyticDB for PostgreSQL. We recommend choosing ADB if you need advanced features such as database management, auditing, and monitoring. ** **Note ** If you have not used ADB storage on Alibaba Cloud Model Studio, go to the Create Knowledge Base page, set the vector storage class to ADB-PG, and follow the on-screen instructions to grant the required permissions. If you set this parameter to ADB, you must specify the `SinkInstanceId` and `SinkRegion` parameters. Valid values: - BUILT_IN : BUILT_IN - ADB : ADB | BUILT_IN |
| SinkInstanceId | string | No | The ID of the AnalyticDB for PostgreSQL instance. This parameter is required only when `SinkType` is set to ADB. Go to the AnalyticDB for PostgreSQL instance list page to obtain this ID. | gp-bp32109xxxx |
| SinkRegion | string | No | The region where the AnalyticDB for PostgreSQL instance is located. This parameter is required only when `SinkType` is set to ADB. You can call the DescribeRegions operation to obtain a list of regions. | cn-hangzhou |
| Columns | array | No | ** **Note ** This parameter is not yet available. Do not specify it. | |
| object | No | ** **Note ** This parameter is not yet available. Do not specify it. | ||
| Column | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | school |
| IsRecall | boolean | No | ** **Note ** This parameter is not yet available. Do not specify it. | true |
| IsSearch | boolean | No | ** **Note ** This parameter is not yet available. Do not specify it. | true |
| Name | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | School |
| Type | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | string |
| Description | string | No | The description of the knowledge base. The description can be 0 to 1,000 English or Chinese characters in length. The default value is empty. | Enterprise doc lib includes important materials |
| metaExtractColumns | array | No | The metadata extraction configuration. Metadata is a series of additional attributes related to unstructured data content. These attributes are integrated into text chunks as key-value pairs. For more information, see Knowledge bases. | |
| object | No | |||
| Key | string | No | The metadata field. The field name can be 1 to 50 characters in length and must consist of letters or underscores. If you specify this parameter, you must also specify the `Value` and `Type` parameters. | author |
| Value | string | No | The value of the metadata field. | Tim |
| Type | string | No | The method for obtaining the value of the metadata field. Valid values: - constant: Constant. - variable: Variable. - custom_prompt: Large Language Model (LLM). - regular: Regular expression. - keywords: Keyword search. Valid values: - constant : Constant extraction - keywords : Keyword extraction - custom_prompt : Large Language Model - variable : Variable extraction - regular : Regular expression | constant |
| Desc | string | No | The Chinese description of the metadata field. The description can be 0 to 1,000 characters in length and can contain Chinese characters, letters, digits, underscores (_), hyphens (-), periods (.), and colons (😃. The default value is empty. | Author name |
| EnableLlm | boolean | No | If enabled, the metadata field and its value are used along with the text chunk content in the answer generation process of the LLM. Valid values: - true: Enabled. - false: Disabled. The default value is false. Valid values: - true : Enabled - false : Disabled | false |
| EnableSearch | boolean | No | If enabled, the metadata field and its value are used along with the text chunk content in the knowledge base retrieval process. Valid values: - true: Enabled. - false: Disabled. The default value is false. Valid values: - true : Enabled - false : Disabled | false |
| enableHeaders | boolean | No | Specifies whether to treat the first row of all .xlsx and .xls files as the table header and append it to each text chunk. This prevents the LLM from treating the header as a regular data row. ** **Note ** We recommend enabling this feature only when all imported files are in .xlsx or .xls format and contain a header. Otherwise, do not enable it. Valid values: - true: Enabled. - false: Disabled. If you do not specify this parameter, this feature is disabled by default. Valid values: - true : Enabled - false : Disabled | false |
| chunkMode | string | No | ** **Note ** This parameter is not yet available. Do not specify it. Valid values: - regex : Chunk by regular expression - length : Chunk by length - h1 : Chunk by level-1 heading - h2 : Chunk by level-2 heading - page : Chunk by page | regex |
| EnableRewrite | boolean | No | Specifies whether to enable multi-turn conversation rewriting. Valid values: - true: Enabled. - false: Disabled. If you do not specify this parameter, this feature is enabled by default. Valid values: - true : Enabled - false : Disabled | true |
| CreateIndexType | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | standard |
| pipelineCommercialType | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | standard |
| pipelineCommercialCu | integer | No | ** **Note ** This parameter is not yet available. Do not specify it. | 1 |
| pipelineRetrieveRateLimitStrategy | string | No | ** **Note ** This parameter is not yet available. Do not specify it. | downgrade |
| knowledgeType | string | No | The data source code. This parameter is required when creating a data query knowledge base and is used with `table` and `database`. ** **Note ** - This operation does not support associating custom databases. Use the Alibaba Cloud Model Studio console to create them. | document |
| RerankMode | string | No | The name of the data table. This parameter is required when creating a data query knowledge base. The data table must exist in the data source specified by `connectId` or `datasourceCode`. Valid values: - similar: : similar: Similarity mode. - custom: : custom: Custom mode. - : qa: (Default) Q&A mode. | qa |
| RerankInstruct | string | No | The name of the database. This parameter is required when creating a data query knowledge base. The database must exist in the data source specified by `datasourceCode`. |
Response elements
| Element | Type | Description | Example |
|---|---|---|---|
| object | Schema of Response | ||
| Code | string | Error status code | Forbidden |
| Data | object | The business data returned when the request is successful. | |
| Id | string | The ID of the knowledge base, also known as `IndexId`. This is the unique identifier of the created knowledge base. ** **Note ** Keep this value safe. It will be used for all subsequent API operations related to this knowledge base. | jkurxhxxxx |
| Message | string | Error message | Invalid input, variable name is missing |
| RequestId | string | Request ID | 17204B98-xxxx-4F9A--2446A84821CA |
| Status | string | The status code returned by the interface. | "200" |
| Success | boolean | Indicates whether the request was successful. Possible values: - true: Successful - false: Failed | true |
Examples
Success response
JSON format
HELPCODEESCAPE-json
{
"Code": "Forbidden",
"Data": {
"Id": "jkurxhxxxx"
},
"Message": "Invalid input, variable name is missing",
"RequestId": "17204B98-xxxx-4F9A--2446A84821CA",
"Status": "\"200\"",
"Success": true
}Error codes
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.