Skip to content

Commit e77ae52

Browse files
Roopan-MicrosoftAjitPadhi-MicrosoftPavan-Microsoftross-p-smithgpickett
authored
fix: merging dev changes to main branch (#1883)
Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Ajit Padhi <[email protected]> Co-authored-by: Pavan-Microsoft <[email protected]> Co-authored-by: Ross Smith <[email protected]> Co-authored-by: gpickett <[email protected]> Co-authored-by: Francia Riesco <[email protected]> Co-authored-by: Francia Riesco <[email protected]> Co-authored-by: Prajwal D C <[email protected]> Co-authored-by: Harmanpreet-Microsoft <[email protected]> Co-authored-by: UtkarshMishra-Microsoft <[email protected]> Co-authored-by: Priyanka-Microsoft <[email protected]> Co-authored-by: Prasanjeet-Microsoft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kiran-Siluveru-Microsoft <[email protected]> Co-authored-by: Prashant-Microsoft <[email protected]> Co-authored-by: Rohini-Microsoft <[email protected]> Co-authored-by: Avijit-Microsoft <[email protected]> Co-authored-by: RaviKiran-Microsoft <[email protected]> Co-authored-by: Somesh Joshi <[email protected]> Co-authored-by: Himanshi Agrawal <[email protected]> Co-authored-by: pradeepjha-microsoft <[email protected]> Co-authored-by: Harmanpreet Kaur <[email protected]> Co-authored-by: Bangarraju-Microsoft <[email protected]> Co-authored-by: Harsh-Microsoft <[email protected]> Co-authored-by: Kanchan-Microsoft <[email protected]> Co-authored-by: Cristopher Coronado <[email protected]> Co-authored-by: Cristopher Coronado Moreira <[email protected]> Co-authored-by: Vamshi-Microsoft <[email protected]> Co-authored-by: Thanusree-Microsoft <[email protected]> Co-authored-by: Niraj Chaudhari (Persistent Systems Inc) <[email protected]> Co-authored-by: Rohini-Microsoft <[email protected]>
1 parent f0c760f commit e77ae52

36 files changed

+35372
-11329
lines changed

.github/workflows/ci.yml

Lines changed: 257 additions & 29 deletions
Large diffs are not rendered by default.

.github/workflows/comment_coverage.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ jobs:
3939
return response.data[0]?.number ?? "";
4040
retries: 3
4141
- name: Comment coverage
42-
uses: MishaKav/pytest-coverage-comment@81882822c5b22af01f91bd3eacb1cefb6ad73dc2
42+
uses: MishaKav/pytest-coverage-comment@13d3c18e21895566c746187c9ea74736372e5e91
4343
with:
4444
pytest-xml-coverage-path: coverage.xml
4545
junitxml-path: coverage-junit.xml

.github/workflows/test-automation.yml

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,29 @@
11
name: Test Automation CWYD
22
on:
3-
push:
4-
branches:
5-
- main
6-
- dev
7-
paths:
8-
- 'tests/e2e-test/**'
9-
schedule:
10-
- cron: '0 13 * * 3' # Runs at 1 PM UTC once a week on Wednesday
11-
workflow_dispatch:
3+
workflow_call:
4+
inputs:
5+
web_url:
6+
required: true
7+
type: string
8+
admin_url:
9+
required: true
10+
type: string
1211

13-
env:
14-
web_url: ${{ vars.CWYD_WEB_URL }}
15-
admin_url: ${{ vars.CWYD_ADMIN_URL }}
16-
accelerator_name: "Chat with your Data"
1712

1813
jobs:
1914
test:
2015
runs-on: ubuntu-latest
16+
env:
17+
web_url: ${{ inputs.web_url }}
18+
admin_url: ${{ inputs.admin_url }}
19+
accelerator_name: "Chat with your Data"
20+
2121
steps:
2222
- name: Checkout repository
2323
uses: actions/checkout@v4
2424

2525
- name: Set up Python
26-
uses: actions/setup-python@v4
26+
uses: actions/setup-python@v5
2727
with:
2828
python-version: '3.13'
2929

Makefile

Lines changed: 104 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -57,14 +57,114 @@ azd-login: ## 🔑 Login to Azure with azd and a SPN
5757
@echo -e "\e[34m$@\e[0m" || true
5858
@azd auth login --client-id ${AZURE_CLIENT_ID} --client-secret ${AZURE_CLIENT_SECRET} --tenant-id ${AZURE_TENANT_ID}
5959

60-
deploy: azd-login ## 🚀 Deploy everything to Azure
60+
azd-login: ## 🔑 Login to Azure with azd and a SPN
61+
@echo -e "\e[34m$@\e[0m" || true
62+
@azd auth login --client-id ${AZURE_CLIENT_ID} --client-secret ${AZURE_CLIENT_SECRET} --tenant-id ${AZURE_TENANT_ID}
63+
64+
# Fixed Makefile section for deploy target
65+
# Fixed Makefile section for deploy target
66+
deploy: azd-login ## Deploy everything to Azure
6167
@echo -e "\e[34m$@\e[0m" || true
6268
@azd env new ${AZURE_ENV_NAME}
63-
@azd env set AZURE_APP_SERVICE_HOSTING_MODEL code --no-prompt
69+
70+
# Provision and deploy
6471
@azd provision --no-prompt
65-
@azd deploy web --no-prompt
66-
@azd deploy function --no-prompt
72+
@azd deploy web --no-prompt || true
73+
@azd deploy function --no-prompt || true
6774
@azd deploy adminweb --no-prompt
75+
@azd env get-values > .env.temp
76+
@cat .env.temp
77+
78+
@sleep 30
79+
@azd show --output json > deploy_output.json || echo "{}" > deploy_output.json
80+
@echo "=== deploy_output.json contents ==="
81+
@cat deploy_output.json | jq . || cat deploy_output.json
82+
83+
# Extract URLs
84+
@echo "=== Extracting URLs using multiple methods ==="
85+
@azd show 2>&1 | tee full_deployment_output.log
86+
@jq -r '.services.web?.project?.hostedEndpoints?[0]?.url // ""' deploy_output.json > frontend_url.txt || echo "" > frontend_url.txt
87+
@jq -r '.services.adminweb?.project?.hostedEndpoints?[0]?.url // ""' deploy_output.json > admin_url.txt || echo "" > admin_url.txt
88+
@grep -oE "https://app-[a-zA-Z0-9-]*\.azurewebsites\.net/" full_deployment_output.log | grep -v admin | head -1 >> frontend_url.txt || true
89+
@grep -oE "https://app-[a-zA-Z0-9-]*-admin\.azurewebsites\.net/" full_deployment_output.log | head -1 >> admin_url.txt || true
90+
@sort frontend_url.txt | uniq | grep -v '^$$' | head -1 > frontend_url_clean.txt && mv frontend_url_clean.txt frontend_url.txt || echo "" > frontend_url.txt
91+
@sort admin_url.txt | uniq | grep -v '^$$' | head -1 > admin_url_clean.txt && mv admin_url_clean.txt admin_url.txt || echo "" > admin_url.txt
92+
93+
@echo "=== URL Extraction Results ==="
94+
@FRONTEND_URL=$$(cat frontend_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
95+
ADMIN_URL=$$(cat admin_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
96+
echo "Frontend URL: $$FRONTEND_URL"; \
97+
echo "Admin URL: $$ADMIN_URL"
98+
99+
@echo "=== Final Deployment Status ==="
100+
@echo "Frontend URL:" && cat frontend_url.txt || echo "Not available"
101+
@echo "Admin URL:" && cat admin_url.txt || echo "Not available"
102+
@echo ""
103+
@echo "🚀 Deployment completed!"
104+
@echo "⏰ Authentication will be disabled via GitHub Actions pipeline."
105+
@echo "🔄 Check the pipeline logs for authentication disable status."
106+
@echo "=== Extracting PostgreSQL Host Endpoint ==="
107+
@azd env get-values > .env.temp 2>/dev/null || echo "" > .env.temp
108+
109+
@PG_HOST_VAL=$$(grep '^AZURE_POSTGRESQL_HOST_NAME=' .env.temp | cut -d'=' -f2 | tr -d '"' | xargs); \
110+
if [ -z "$$PG_HOST_VAL" ]; then \
111+
echo "❌ PostgreSQL host not found in .env.temp. Using fallback localhost"; \
112+
PG_HOST_VAL="localhost"; \
113+
else \
114+
echo "✅ PostgreSQL host extracted from .env.temp: $$PG_HOST_VAL"; \
115+
fi; \
116+
echo "$$PG_HOST_VAL" > pg_host.txt
117+
118+
119+
120+
121+
122+
@echo "=== PostgreSQL Configuration ==="
123+
@echo "Username: admintest (hardcoded)"
124+
@echo "Database: postgres (hardcoded)"
125+
@echo "Port: 5432 (hardcoded)"
126+
@echo "Host: $$(cat pg_host.txt 2>/dev/null || echo 'Not available')"
127+
@echo "Password: Initial_0524 (hardcoded)"
128+
129+
# Helper target to check current authentication status
130+
check-auth:
131+
@echo "=== Checking Authentication Status ==="
132+
@FRONTEND_URL=$$(cat frontend_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
133+
ADMIN_URL=$$(cat admin_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
134+
if [ -n "$$FRONTEND_URL" ]; then \
135+
echo "Testing Frontend: $$FRONTEND_URL"; \
136+
HTTP_CODE=$$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 10 "$$FRONTEND_URL" 2>/dev/null || echo "000"); \
137+
echo "Frontend HTTP Status: $$HTTP_CODE"; \
138+
fi; \
139+
if [ -n "$$ADMIN_URL" ]; then \
140+
echo "Testing Admin: $$ADMIN_URL"; \
141+
HTTP_CODE=$$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 10 "$$ADMIN_URL" 2>/dev/null || echo "000"); \
142+
echo "Admin HTTP Status: $$HTTP_CODE"; \
143+
fi
144+
145+
# Helper target to manually disable authentication (for debugging)
146+
disable-auth-manual:
147+
@echo "=== Manually Disabling Authentication ==="
148+
@echo "This target requires Azure CLI to be logged in manually"
149+
@FRONTEND_URL=$$(cat frontend_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
150+
ADMIN_URL=$$(cat admin_url.txt 2>/dev/null | tr -d '\n\r' | xargs); \
151+
export FRONTEND_WEBSITE_URL="$$FRONTEND_URL"; \
152+
export ADMIN_WEBSITE_URL="$$ADMIN_URL"; \
153+
if [ -f "disable_auth.sh" ]; then \
154+
chmod +x disable_auth.sh && ./disable_auth.sh; \
155+
else \
156+
echo "ERROR: disable_auth.sh not found"; \
157+
exit 1; \
158+
fi
159+
160+
disable-auth-fixed:
161+
@echo "=== Using Fixed Authentication Disable Script ==="
162+
@if [ -f "disable_auth_fixed.sh" ]; then \
163+
chmod +x disable_auth_fixed.sh && ./disable_auth_fixed.sh; \
164+
else \
165+
echo "ERROR: disable_auth_fixed.sh not found"; \
166+
exit 1; \
167+
fi
68168

69169
destroy: azd-login ## 🧨 Destroy everything in Azure
70170
@echo -e "\e[34m$@\e[0m" || true

code/backend/batch/utilities/helpers/env_helper.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ def __load_config(self, **kwargs) -> None:
166166
"Unsupported DATABASE_TYPE. Please set DATABASE_TYPE to 'CosmosDB' or 'PostgreSQL'."
167167
)
168168

169-
self.AZURE_AUTH_TYPE = os.getenv("AZURE_AUTH_TYPE", "keys")
169+
self.AZURE_AUTH_TYPE = os.getenv("AZURE_AUTH_TYPE", "rbac")
170170
# Azure OpenAI
171171
self.AZURE_OPENAI_RESOURCE = os.getenv("AZURE_OPENAI_RESOURCE", "")
172172
# Fetch AZURE_OPENAI_MODEL_INFO from environment
@@ -234,6 +234,7 @@ def __load_config(self, **kwargs) -> None:
234234
self.AZURE_COMPUTER_VISION_VECTORIZE_IMAGE_MODEL_VERSION = os.getenv(
235235
"AZURE_COMPUTER_VISION_VECTORIZE_IMAGE_MODEL_VERSION", "2023-04-15"
236236
)
237+
self.FUNCTION_KEY = os.getenv("FUNCTION_KEY", "")
237238

238239
# Initialize Azure keys based on authentication type and environment settings.
239240
# When AZURE_AUTH_TYPE is "rbac", azure keys are None or an empty string.
@@ -242,6 +243,7 @@ def __load_config(self, **kwargs) -> None:
242243
self.AZURE_OPENAI_API_KEY = ""
243244
self.AZURE_SPEECH_KEY = None
244245
self.AZURE_COMPUTER_VISION_KEY = None
246+
self.FUNCTION_KEY = self.secretHelper.get_secret("FUNCTION_KEY")
245247
else:
246248
self.AZURE_SEARCH_KEY = self.secretHelper.get_secret("AZURE_SEARCH_KEY")
247249
self.AZURE_OPENAI_API_KEY = self.secretHelper.get_secret(
@@ -269,7 +271,6 @@ def __load_config(self, **kwargs) -> None:
269271
os.environ["OPENAI_API_VERSION"] = self.OPENAI_API_VERSION
270272
# Azure Functions - Batch processing
271273
self.BACKEND_URL = os.getenv("BACKEND_URL", "http://localhost:7071")
272-
self.FUNCTION_KEY = os.getenv("FUNCTION_KEY")
273274
self.AzureWebJobsStorage = os.getenv("AzureWebJobsStorage", "")
274275
self.DOCUMENT_PROCESSING_QUEUE_NAME = os.getenv(
275276
"DOCUMENT_PROCESSING_QUEUE_NAME", "doc-processing"

code/create_app.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,11 @@ def conversation_with_data(conversation: Request, env_helper: EnvHelper):
209209
env_helper.AZURE_SEARCH_CONTENT_VECTOR_COLUMN
210210
],
211211
"title_field": env_helper.AZURE_SEARCH_TITLE_COLUMN or None,
212+
"source_field": env_helper.AZURE_SEARCH_SOURCE_COLUMN
213+
or None,
214+
"text_field": env_helper.AZURE_SEARCH_TEXT_COLUMN or None,
215+
"layoutText_field": env_helper.AZURE_SEARCH_LAYOUT_TEXT_COLUMN
216+
or None,
212217
"url_field": env_helper.AZURE_SEARCH_FIELDS_METADATA
213218
or None,
214219
"filepath_field": (
@@ -521,7 +526,6 @@ def speech_config():
521526
if response.status_code == 200:
522527
return {
523528
"token": response.text,
524-
"key": speech_key,
525529
"region": env_helper.AZURE_SPEECH_SERVICE_REGION,
526530
"languages": env_helper.AZURE_SPEECH_RECOGNIZER_LANGUAGES,
527531
}

0 commit comments

Comments
 (0)